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United  states  General  accounting  office 

WASHINGTON,  O.C.  20548 


TEOERAL  PERSONNEL  AND 
COMPENSATION  DIVISION 


B-205861 


The  Honorable  John  O.  Marsh,  Jr. 

The  Secretary  of  the  Army 

Dear  Mr.  Secretary: 

Attention:  The  Inspector  General 

DAIG-AI 

Pursuant  to  a  request  from  Congressman  Leon  E.  Panetta,  we 
examined  the  management  operations  of  the  Defense  Language  Insti¬ 
tute,  Foreign  Language  Center,  at  Monterey,  California.  The  re¬ 
port  discusses  what  we  believe  are  the  significant  internal  prob¬ 
lems  which  diminish  the  overall  effectiveness  of  language  training 
at  the  Institute. 

The  report  contains  recommendations  which  require  specific 
action  on  your  part.  As  you  know,  section  236  of  the  Legislative 
Reorganization  Act  of  1970  requires  the  head  of  a  Federal  agency 
to  submit  a  written  statement  on  actions  taken  on  our  recommenda¬ 
tions  to  the  House  Committee  on  Government  Operations  and  the 
Senate  Committee  on  Governmental  Affairs  not  later  than  60  days 
after  the  date  of  the  report.  A  written  statement  must  also  be 
sent  to  the  House  and  Senate  Committees  on  Appropriations  with 
the  agency's  first  request  for  appropriations  made  more  than 
60  days  after  the  date  of  the  report. 

We  are  sending  copies  of  this  report  to  the  Director,  Office 
of  Management  and  Budget,  and  to  the  Chairmen,  House  Committee  on 
Government  Operations,  Senate  Committee  on  Governmental  Affairs, 
House  and  Senate  Committees  on  Appropriations,  House  and  Senate 
Select  Committees  on  Intelligence,  and  House  and  Senate  Commit¬ 
tees  on  Armed  Services. 


Sincerely  yours. 


GENERAL  ACCOUNTING  OFFICE 
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OF  THE  ARMY 


WEAKNESSES  IN  THE  RESIDENT 
LANGUAGE  TRAINING  SYSTEM  OF 
DEFENSE  LANGUAGE  INSTITUTE 
AFFECT  THE  QUALITY  OF  TRAINED 
LINGUISTS 


DIGEST 


"The  Defense  Language  Institute’s  Foreign  Lan¬ 
guage  Center  is  responsible  for  providing 
foreign  language  training  to  military  person¬ 
nel  who  are  being  prepared  for  intelligence 
activities.  The  Institute's  mission  is  to 
conduct  and  supervise  language  training  for 
these  personnel  and  to  provide  technical  sup¬ 
port  for  all  other  foreign  language  training 
conducted  for  the  services  except  for  mili¬ 
tary  academies  and  overseas  schools.  GAO''- 
condueted  this  review  at  the  request  ofr 
"Corrgr e a sman-  Leon  E.  Panetta .  GAO  ’  s  ob  jec-  jl  l 

tives  were  to  identify  and  analyze  signif¬ 
icant  internal  Defense  Language  Institute 
problems  that  diminish  the  overall  effective¬ 
ness  of  language  training  at  the  Institute. 

GAO  reviewed  the  management  of  training  at 
the  Institute  and  concluded  that  changes  are 
necessary  to  improve  the  quality  of  language 
instruction.  More  specifically,  the  Insti-  I 

tute  needs  to  (1)  replace  outdated  materials 
in  basic  courses,  (2)  upgrade  the  management 
of  classroom  instruction,  and  (3)  better  as-  / 
sess  the  effectiveness  of  its  training  mission . 

I  < 

EFFORTS  TO  DEVELOP  LJ>1 

QUALITY  COURSE  MATERIALS  /  A 

HAVE  BEEN  UNPRODUCTIVE  /  ' 

iDis  t 

The  Institute  has  made  little  progress  in  im -i  ; 

proving  the  quality  of  current  course  mate-  /  S) , 
rials.  It  has  expended  159  staff-years  at  /  //  i 
a  cost  of  about  $4.2  million  but  has  not  pro^~L^I 
duced  needed  basic  course  materials.  Defense 
officials  said,  however,  that  new  course  mate¬ 
rials  would  be  forthcoming  during  1982. 
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The  lack  of  progress  has  been  caused,  in  part, 
by  the  Institute's  failure  to  (1)  effectively 
set  course  development  priorities,  (2)  properly 
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implement  prescribed  course  development 
procedures,  (3)  adequately  monitor  progress  of 
course  development  projects,  (4)  fully  explore 
the  potential  of  using  commercial  textbooks, 
and  (5)  effectively  use  contracting  to  obtain 
needed  course  materials.  (See  p.  4.) 

CHANGES  IN  MANAGEMENT  OF 
CLASSROOM  INSTRUCTION  SHOULD 
IMPROVE  TRAINING  QUALITY 

Inadequate  management  of  classroom  instruction 
could  be  adversely  affecting  the  quality  of 
classroom  training.  Specifically,  GAO  found 
that: 

— An  officially  approved  and  accepted  teaching 
methodology  was  lacking . 

— Instructor  training  was  limited. 

— Instructors  were  not  being  properly  evalu¬ 
ated  by  supervisors. 

— Response  to  and  followup  on  training  recom¬ 
mendations  were  poor. 

— Technical  language  assistants  had  not  been 
effectively  used.  (See  p.  11.) 

ASSESSMENT  OF  TRAINING  EFFECTIVENESS 
IS  NOT  CONCLUSIVE 


The  Institute  cannot  conclusively  assess  the 
proficiency  of  its  graduates  or  the  effective¬ 
ness  of  its  training  system  on  the  basis  of 
its  existing  evaluation  processes.  It  needs 
to  formulate  a  cohesive  policy  statement  on 
its  training  objectives  and  standards  in  order 
to  train  students  to  desired  user  proficiency 
levels.  The  lack  of  clear  objectives  and 
standards  causes  confusion  over  what  to  evalu¬ 
ate  or  what  the  proficiency  level  of  graduates 
should  be.  (See  p.  17.) 

RECOMMENDATIONS 


GAO  recommends  that  the  Secretary  of  the  Army 
direct  the  Commandant  of  the  Institute  to: 

— Develop  resident  basic  courses  using  commer¬ 
cially  available  materials  whenever  these 


ii 


can  be  adapted  at  less  cost  and  in  less 
time  than  in-house  development  effort. 

— Establish  controls  over  course  development 
projects  which  provide  the  means  to  assess 
progress  against  specified  target  dates. 

— Develop  and  distribute  a  standard  schoolwide 
training  methodology  for  use  in  all  language 
departments . 

— Require  all  newly  hired  instructors  to  com¬ 
plete  both  phases  of  the  basic  instructor¬ 
training  course.  In  addition,  instructors 
should  be  encouraged  to  seek  out  additional 
training  to  improve  their  instructional 
abilities  as  part  of  the  individual  develop¬ 
ment  programs. 

— Establish  procedures  to  carry  out  the  rein¬ 
stated  policy  for  supervisory  classroom 
visits  and  hold  supervisors  accountable  for 
routinely  observing  instructor  classroom 
behavior . 

— Establish  realistic  training  objectives  based 
on  mission  requirement  and  use  the  Defense 
Language  Proficiency  Test  to  measure  students’ 
performance  in  satisfaction  of  these  objec¬ 
tives  and  require  that  students  pass  the  test 
as  a  condition  for  graduation. 

— Develop  a  Defense  Language  Proficiency  Test 
that  will  measure  student  proficiency  of 
the  objectives  and  standards  established 
by  the  Institute. 

AGENCY  COMMENTS 


The  Department  of  Defense  commented  that  it 
shares  a  common  interest  with  GAO  in  the  effec¬ 
tiveness  of  training  at  the  Institute  and  stated 
that  several  years  ago  it  observed  many  of  the 
conditions  reported  by  GAO.  Defense  officials 
reported  that,  since  GAO’s  audit  efforts  were  com¬ 
pleted,  there  has  been  a  high  level  of  accom¬ 
plishment  and  that  many  problems  noted  in  this 
report  either  have  been  or  are  being  addressed.. 

GAO  modified  some  of  the  proposed  recommendations 
in  its  draft  report  as  a  result  of  Defense’s 
comments  regarding  actions  taken  or  underway. 

iii 
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These  modifications  are  addressed  in  the 
recommendation  and  agency  comment  sections 
of  chapters  2,  3,  and  4.  (See  pages  10,  IS, 
and  21.) 
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CHAPTER  1 


INTRODUCTION 


The  Defense  Language  Institute  (DLI),  Foreign  Language  Center, 
in  Monterey,  California,  was  established  to  provide  foreign  lan¬ 
guage  training  for  the  entire  Department  of  Defense  (DOD).  DLI 
is  under  the  administrative  control  of  the  Department  of  the  Army 
and  more  specifically  under  that  of  the  Army  Training  and  Doctrine 
Command  (TRADOC)  at  Fort  Monroe,  Virginia.  DLI ' s  mission  is  very 
important  because  its  products,  foreign-language-trained  personnel, 
support  the  defense  intelligence  mission.  Although  linguists  con¬ 
stitute  a  very  small  percentage  of  total  military  personnel,  DOD 
considers  them  an  essential  element  for  preserving  national 
security. 

DLI  conducts  full-time  intensive  foreign  language  training  and 
provides  technical  control  for  all  other  foreign  language  training 
conducted  in  DOD,  except  for  military  academies  and  overseas  DOD- 
operated  schools.  The  instructional  program  is  uniquely  geared  to 
the  needs  of  defense,  and  most  DLI  students  are  active  duty  enlisted 
service  members  who  eventually  are  assigned  to  defense  intelligence 
jobs.  DLI 1 s  basic  resident  courses,  those  taught  at  the  Presidio 
of  Monterey  and  at  Lackland  Air  Force  Base,  Texas,  are  aimed  at 
developing  working  level  competencies  in  listening  comprehension, 
reading,  speaking,  and  writing.  DLI  also  develops  nonresident  lan¬ 
guage  training  programs  for  people  in  military  field  units  and 
elsewhere  to  regain,  maintain,  or  enhance  language  proficiency  by 
jobs  and  missions.  In  addition,  DLI  is  responsible  for 

— developing  and  maintaining  instructional  material  for  both 
the  resident  and  nonresident  programs; 

— planning  for  faculty  development; 

— employing,  training,  and  maintaining  qualified  subject 
matter  experts  in  job  and  task  analysis,  testing,  evalu¬ 
ation,  curriculum  development,  and  instruction  in  foreign 
languages;  and 

— exercising  quality  control  over  the  foreign  language 
program  by  providing  standards  and  tests  to  measure 
language  proficiency. 

DLI  currently  provides  training  in  about  37  major  languages 
and  dialects;  it  relies  almost  solely  on  native-speaking  instruc¬ 
tors.  DLI  exercises  very  little  real  control  over  the  numbers 
and  timing  of  students  scheduled  for  language  training  or  the 
languages  to  be  taught.  User  agencies  determine  terminal  learn¬ 
ing  objectives  (required  language  skills)  and,  in  conjunction 
with  DLI,  establish  the  length  of  time  students  will  be  in  train¬ 
ing.  Except  for  the  Army,  the  services  also  maintain  administra¬ 
tive  control  over  their  own  students  while  at  DLI.  Coordination 
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between  DLI  and  user  agencies  is  done  primarily  through  an  annual 
program  review  at  the  beginning  of  each  calendar  year.  Staffing 
at  DLI  includes  some  350  military  personnel  and  a  civilian  work 
force  of  850,  of  which  about  600  are  faculty  members.  DLI  teaches 
foreign  languages  to  about  3,500  service  students  per  year  (the 
student  load  averages  about  2,600),  of  which  the  largest  number 
belongs  to  the  Department  of  the  Army.  (App.  II  lists  student 
enrollment  and  number  of  instructors  by  language  as  of  September 
1981.) 

QUALITY  OF  LINGUISTS  AND  TRAINING 
HAS  BEEN  A  LONGSTANDING  CONCERN 


The  quality  of  foreign  language  training  and  the  competence 
of  military  and  civilian  linguists  have  been  longstanding  con¬ 
cerns.  For  example,  we  reported  in  1973  1 /  that  foreign-language¬ 
training  programs  did  not  always  give  personnel  the  proficiency 
required  to  do  their  jobs.  We  also  reported  in  1980  2/  that  DOD 
had  a  large  number  of  language-designated  positions  either  un¬ 
filled  or  not  filled  at  the  required  proficiency  level.  In  addi¬ 
tion,  what  they  learned  was  often  not  specifically  related  to 
the  requirements  of  their  jobs.  User  agencies  also  have  become 
increasingly  vocal  about  the  inability  of  DLI-trained  personnel 
newly  assigned  at  duty  stations  to  perform  basic  linguist  duties. 

In  addition,  DLI ' s  own  evaluations  of  tactical  and  strategic 
intelligence  units,  conducted  in  fiscal  years  1979  and  1981,  con¬ 
firmed  users*  complaints.  A  major  reason  often  cited  by  linguists 
and  their  supervisors  for  language  deficiencies  was  inadequate 
basic  language  training  while  at  DLI. 

OBJECTIVE,  SCOPE,  AND  METHODOLOGY 

At  the  request  of  Congressman  Leon  E.  Panetta,  we  reviewed 
the  operations  and  training  at  DLI.  This  review  was  performed 
in  Monterey  between  January  and  September  1981  in  accordance 
with  our  Office's  current  "Standards  for  Audit  of  Governmental 
Organizations,  Programs,  Activities,  and  Functions."  Our  objec¬ 
tive  was  to  identify  and  analyze  those  significant  internal  prob¬ 
lems  that  diminish  the  overall  effectiveness  of  language  training. 

Congressman  Panetta  agreed  that  we  would  address  only  those 
issues  which  related  directly  to  DLI  training  capabilities  and 


lV"Need  To  Improve  Language  Training  Programs  and  Assignments 
for  U.S.  Government  Personnel  Overseas"  (B-176049,  Jan.  22, 
1973) . 

2/"More  Competence  in  Foreign  Languages  Needed  by  Federal  Per¬ 
sonnel  Working  Overseas"  (ID-80-31,  Apr.  15,  1980). 
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were  within  the  jurisdiction  of  the  Commandant  of  DLI  to  correct. 

The  three  issues  examined  were  the  adequacy  of  (1)  course  develop¬ 
ment  activities,  (2)  management  of  classroom  instruction,  and 
(3)  evaluation  of  graduates  and  training. 

Our  review  included  an  analysis  of  previous  studies  (see 
app.  Ill)  of  DLI  and  an  examination  of  the  fiscal  years  1979 
and  1981  external  field  evaluations  that  DLI  performed.  We  did 
not  verify  the  accuracy  of  findings  reported  by  these  evalua¬ 
tions,  nor  did  we  evaluate  the  methodology  used  in  making  the 
evaluations  or  in  developing  the  respective  findings. 

Our  work  also  included  examinations  of  various  internal  docu¬ 
ments,  such  as  DLI  Instructional  Systems  Audits;  recently  com¬ 
pleted  student  and  faculty  questionnaires;  DLI  regulations  and 
internal  documented  policy  guidance  on  instructional  methodology, 
testing  and  grading,  and  course  development  activities;  and  in¬ 
structors'  training  and  appraisal  records.  We  also  reviewed  stu¬ 
dent  end-of-course  grades  and  compared  them  with  students'  language 
proficiency  test  results  to  determine  if  students  could  attain  the 
level  of  proficiency  required  by  users. 

Interviews  with  department  heads,  supervisors,  instructors, 
and  students  were  confined  to  the  six  largest  language  depart¬ 
ments — Russian,  German,  Korean,  Chinese,  Arabic,  and  Spanish — which 
in  total  account  for  over  90  percent  of  both  faculty  and  students 
at  DLI.  Individuals  interviewed  were  randomly  selected  to  obtain 
a  cross  section  of  opinions.  However,  these  selections  do  not  con¬ 
stitute  a  statistical  sample  and,  therefore,  opinions  expressed  do 
not  necessarily  represent  the  views  of  all  DLI  faculty  and  students. 

Other  internal  problems  at  DLI,  such  as  the  questions  of  com¬ 
petitive  versus  excepted  service  status  for  DLI  faculty,  employee 
morale  and  grievances,  abolishment  of  the  Academic  Senate,  cross- 
cultural  communication  difficulties,  and  organizational  structure 
issues,  were  not  reviewed,  as  agreed  with  Congressman  Panetta. 
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CHAPTER  2 


DLI's  COURSE  DEVELOPMENT  EFFORTS 
HAVE  BEEN  UNPRODUCTIVE 

Students  are  not  receiving  up-to-date  language  instruction. 

DLI  officials,  user  agencies'  representatives,  and  others  ac¬ 
knowledge  that  DLI's  resident  courses  are  outdated,  but  DLI  has 
made  little  progress  in  developing  new  resident  materials  for 
basic  courses.  Between  fiscal  year  1978  and  the  second  quarter 
of  fiscal  year  1981,  DLI  used  about  $4.2  million  and  159  staff- 
years  for  course  development  and  has  yet  to  develop  updated  basic 
course  materials.  DLI's  lack  of  progress  has  been  caused  largely 
by  DLI's  failure  to 

— effectively  set  course  development  priorities  between 
its  resident  and  nonresident  courses, 

— properly  implement  the  Interservice  Procedures  for  In¬ 
structional  Systems  Development  promulgated  by  TRADOC, 

— effectively  monitor  the  progress  of  ongoing  course  de¬ 
velopment  projects, 

— fully  pursue  acquisition  of  commercial  texts  as  an 
alternative  to  in-house  course  development,  and 

— use  contracting  effectively  to  obtain  needed  materials 
and  to  increase  the  use  of  in-house  resources. 

DLI  officials  acknowledged  that  the  Directorate  of  Training 
Development  had  not  completely  rewritten  any  resident  basic 
courses.  However,  lack  of  progress  was  attributed  to  the  (1)  de¬ 
velopment  of  nonresident  materials  requested  by  user  agencies  for 
worldwide  use,  (2)  rigorous  and  time-consuming  requirements  of 
TRADOC 's  Interservice  Procedures  for  Instructional  Systems  Develop¬ 
ment,  and  (3)  numerous  delays  in  completing  course  development 
projects  caused  by  project  staffing  difficulties  and  interruptions. 

FAILURE  TO  EFFECTIVELY  SET  PRIORITY 
ON  RESIDENT  BASIC  COURSES  RESULTED  IN 
DISPROPORTIONATE  AMOUNT  OF  RESOURCES 
SPENT  ON  NONRESIDENT  COURSES 


Despite  the  need  for  new  resident  basic  courses,  DLI  has  not 
effectively  set  priority  on  these  projects  and  has  spent  dispropor¬ 
tionate  resources  on  nonresident  course  development. 


DLI  develops  materials  for  both  resident  and  nonresident 
courses.  Resident  training  is  that  training  which  takes  place  at 
the  Presidio  of  Monterey  and  Lackland  Air  Force  Base  and  consists 
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primarily  of  the  basic,  intermediate,  and  advanced  courses. 
Nonresident  training,  as  the  name  implies,  is  designed  for  use 
at  military  activities  where  linguists  are  employed. 

New  resident  basic  course  materials  are 
needed  but  have  not  been  developed 

Many  sources  have  noted  that  DLI's  resident  courses  need 
revision.  in  1979  it  was  reported  that  resident  course  mate¬ 
rials  ranged  in  age  from  4  to  27  years  and  desperately  needed 
attention.  Despite  DLI's  course  development  efforts,  however, 
no  new  resident  courses  have  been  implemented  since  1976.  Fur¬ 
thermore,  DLI's  primary  user  agency,  the  National  Security 
Agency  (NSA),  during  a  special  program  review  conducted  at  DLI 
during  December  1980,  charged  that: 

"While  177  roanyears  have  been  expended  in  course 

development  not  a  single  resident  course  has  been 

completed. " 

DLI  officials  do  not  dispute  the  fact  that  resident  course  mate¬ 
rials  are  outdated. 

DLI  has  no  system  to  effectively 
set  course  development  priorities 

DLI's  course  development  process  depends  upon  obtaining  the 
consensus  of  the  user  agencies  during  the  annual  program  reviews. 
DLI  officials  told  us  there  was  no  formal  list  of  priorities? 
however,  priorities  are  now  stated  within  the  Five-Year  Plan  for 
course  development.  User  agency  officials  told  us  that,  jprior  to 
the  approved  Five-Year  Plan,  DLI  had  been  unable  to  set  clear 
course  development  priorities  because  there  had  been  no  consensus 
among  the  user  agencies  as  to  what  courses  should  have  the  highest 
priority.  For  example,  while  NSA  placed  its  priority  on  resident 
course  development,  two  Army  commands  were  more  concerned  with 
obtaining  nonresident  materials.  In  addition,  the  Marine  Corps, 
while  it  concurred  in  the  need  to  place  priority  on  development 
of  resident  basic  courses,  also  desired  further  development  of 
Training  Extension  Courses.  The  Marine  Corps  later  objected  when 
DLI  curtailed  some  extension  course  development  in  favor  of  basic 
course  projects. 

Disproportionate  amount  of  resources 
have  been  spent  on  nonresident  courses 

DLI  has  not  balanced  the  priority  for  its  course  development 
needs.  A  disportionate  amount  of  resources  have  been  expended 
on  nonresident  courses  in  trying  to  satisfy  the  diverse  needs  of 
user  agencies. 
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Our  computations,  made  from  data  in  DLI's  records,  show  that, 
of  the  159.1  staff-years  expended  for  course  development  between 
fiscal  year  1978  and  the  second  quarter  of  fiscal  year  1981,  only 
33  percent  was  spent  on  resident  courses  while  67  percent  was  spent 
on  nonresident  courses,  as  shown  by  the  following  table. 


Resident  courses: 

Basic  course  development  51.1 
Basic  course  revision  0.6 
Intermediate  and  advanced  course  development  0.9 


Total  52.6 


Nonresident  courses: 

Headstart  course  development  and  revision  16.7 
Gateway  course  development  and  revision  16.2 
Refresher  and  maintenance  course  development  13.2 
Training  extension  course  development  60.5 


Total  106.6 


DLI  HAS  IMPROPERLY  IMPLEMENTED  THE 
INSTRUCTIONAL  SYSTEMS  DEVELOPMENT  APPROACH 

DLI  uses  TRADOC 1 s  instructional  systems  development  approach 
guidance  for  language  course  development.  DLI  has  adopted  the 
approach  because  it  contends  that  it  is  the  best  method  for  de¬ 
veloping  training  that  effectively  meets  user  needs.  However,  we 
found  that  DLI  had  improperly  implemented  this  approach  for  some 
of  its  high-density  basic  courses  now  undergoing  revision. 

DLI  Memorandum  5-2,  "Planning  and  Management  of  Training 
Development  Projects,"  dated  March  15,  1979,  provides  that 
training  development  be  accomplished  in  accordance  with  TRADOC 
Pamphlet  350-30,  "Interservice  Procedures  for  Instructional  Sys¬ 
tems  Development."  The  process,  as  detailed  by  Pamphlet  350-30, 
outlines  five  sequential  phases  in  the  development  of  training 
materials:  analysis,  design,  development,  implementation,  and 
control.  We  found,  however,  that  for  at  least  three  courses  be¬ 
ing  revised — Basic  Russian,  Basic  Chinese  Mandarin,  and  Spanish 
refresher/maintenance — DLI  had  conducted  the  phases  in  the  wrong 
order.  In  all  cases,  the  development  phase  preceded  the  analysis 
phase . 


Lack  of  appropriate  front-end  analysis  before  designing  and 
developing  courses  has  also  been  cited  in  a  previous  external 
evaluation  as  a  roadblock  to  successful  course  development.  A 
1979  TRADOC  Inspector  General  evaluation  noted  that  no  signifi¬ 
cant  improvements  had  been  made  in  the  basic  resident  course 
since  TRADOC ' s  prior  1978  annual  inspection.  According  to  the 
evaluation,  the  primary  problem  hindering  effective  basic 
course  development  was  the  lack  of  analysis  of  the  basic  language 
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requirements  to  determine  what  skills  should  be  taught.  The 
Inspector  General  added  that  DLI ' s  Analysis  Division  lacked 
guidance  on  establishing  priorities  in  order  to  best  use  ex¬ 
tremely  limited  resources. 

A  Training  Development  official  said  DLI  had  deviated  from 
the  instructional  systems  model  because  during  1976  considerable 
emphasis  by  the  Commandant  was  placed  on  the  need  to  update  old 
course  materials.  Consequently,  in  trying  to  expedite  develop¬ 
ment  of  new  materials.  Training  Development  gave  less  attention 
to  analysis  and  design  while  prematurely  focusing  on  development. 

DLI  HAS  NOT  ESTABLISHED  ADEQUATE  CONTROLS 
BY  WHICH  TO  MONITOR  AND  MANAGE  PROJECTS 

DLI  has  not  established  adequate  controls  for  monitoring 
progress  and  managing  the  development  of  course  materials.  Proj¬ 
ect  work  plans  are  constantly  being  revised  to  reflect  the  cur¬ 
rent  situation;  project  status  reports  contain  inaccurate  data; 
and  the  lack  of  a  standard  for  measuring  productivity  has  hindered 
DLI ' s  ability  to  monitor  and  manage  course  development. 

As  a  management  tool,  the  work  plans  are  of  limited  use  be¬ 
cause  of  constant  revisions.  DLI  has  allowed  the  project  teams 
to  revise  their  work  plans  to  reflect  current  estimates,  thereby 
limiting  their  value  as  a  baseline  from  which  to  measure  vari¬ 
ances,  assess  the  reasons  for  variances,  and  make  needed  correc¬ 
tions.  We  were  told  that  DLI  had  not  required  teams  to  conform 
with  realistic  work  plans  because  the  project  officers  were  re¬ 
luctant  to  commit  themselves  to  milestones.  In  addition,  offi¬ 
cials  stated  that  resource  and  staffing  priorities  were  so  er¬ 
ratic  that  realistic  planning  was  meaningless.  Because  of  the 
absence  of  records,  we  could  not  determine  the  amount  of  slippage 
the  original  work  plans  had  undergone. 

Project  status  reports  are  another  management  tool.  Accord¬ 
ing  to  DLI  Memo  5-2,  these  reports  should  establish  and  maintain 
continuous  records  on  cost,  time,  manpower  use,  work  accomplish¬ 
ments,  and  developmental  problems.  They  should  also  help  man¬ 
agers  to  (1)  project  future  developmental  resource  requirements, 

(2)  reach  make  or  buy  decisions,  and  (3)  perform  problem-solving 
analyses.  However,  the  reports,  cannot  measure  the  progress  of 
development  because  of  changing  work  plans  as  discussed  above, 
nor  do  they  accurately  report  the  staff-hours  charged  to  projects. 
In  a  sample  of  11  of  the  20  projects  ongoing  during  the  second 
quarter  of  fiscal  year  1981,  the  staff-hours  charged  for  each  pro¬ 
ject  on  the  reports  did  not  agree  with  those  on  DLI ’ s  computer 
system.  The  discrepancy  ranged  from  21.9  to  180.9  staff-hours. 

In  addition,  DLI  has  not  developed  or  used  performance  stand¬ 
ards  to  measure  the  productivity  of  its  project  teams.  NSA  uses 
a  6:1  ratio;  that  is,  the  number  of  staff-hours  required  to  develop 
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materials  for  1  hour  of  classroom  instruction,  as  a  standard  for 
developing  its  language  courses.  While  DLI  argued  that  NSA  1  s 
ratio  was  unrealistically  low,  DLI  still  has  not  seriously  tried 
to  develop  its  own  standards.  It  has  been  suggested  that  DLI  ob¬ 
tain  additional  staff  to  develop  standards;  however,  action  on  this 
suggestion  is  pending  the  results  of  a  planned  staffing  survey. 

DLI  HAS  NOT  FULLY  EXPLORED  THE  POTENTIAL 
OF  USING  COMMERCIAL  TEXTBOOKS 


Usage  of  commercial  texts  has  been  minimal  despite  DLI's 
policy  requiring  such  materials  to  be  evaluated  and  used  whenever 
justified  on  a  cost,  quality,  or  timing  basis.  We  could  not  find, 
nor  could  officials  provide  records  to  indicate,  that  DLI  had 
formally  evaluated  or  incorporated  commercial  texts  before  initi¬ 
ating  costly  and  time-consuming  in-house  development.  We  were 
told  that,  although  project  teams  reviewed  commercial  texts,  DLI 
had  not  documented  the  evaluation  process,  nor  had  it  provided 
specific  guidance  to  the  teams  on  the  content,  methodology,  or 
extent  of  the  evaluations. 

For  example,  "Deutsch  activ, "  a  German  textbook,  was  re¬ 
viewed  by  DLI  staff  and  was  said  to  be  excellent  for  its  superior 
use  of  communicative  skills.  However,  a  formal  evaluation  com¬ 
paring  the  text  to  DLI  needs  and  a  quantitative  analysis  of  what 
it  would  cost  to  adapt  and  use  the  textbook  at  DLI  were  never 
performed.  Regardless,  DLI  awarded  a  contract  for  $25,460  for 
initial  development  of  the  German  Basic  Course.  The  contract 
was  not  successful,  and  DLI  is  now  trying  to  develop  the  German 
Basic  Course  in-house  using  portions  of  the  "Deutsch  activ"  text, 
pending  an  agreement  with  the  German  publisher. 

DLI  staff  have  raised  several  objections  to  using  commercial 
texts.  We  were  told  that  commercial  texts  were  geared  to  a  differ¬ 
ent  audience,  they  lacked  military  "flavor"  or  terminology,  or 
copyright  and  availability  problems  would  interfere.  These  objec¬ 
tions,  however,  have  been  discounted  by  user  agencies  and  other 
DLI  staff  for  the  following  reasons: 

— A  good  basic  text  could  serve  as  the  framework  for  a  course 
with  additional  exercises  and  other  supplemental  materials 
to  provide  the  intensity  needed  by  DLI. 

— Basic  courses  are  not  military  specific  until  the  end,  and 
military  terminology  could  easily  be  added. 

— NSA  and  the  Foreign  Service  Institute  (FSI)  both  use  com¬ 
mercial  texts  extensively  for  their  language  courses. 
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DLI  HAS  HAD  POCR  RESULTS  IN  CONTRACTING 
FOR  BASIC  COURSE  DEVELOPMENT 


DLI 1 s  efforts  to  contract  out  basic  course  development  have 
not  been  successful.  Officials  acknowledge  that  contracts  for 
course  development  between  1969  and  1975  produced  little  usable 
materials,  and  no  completed  basic  courses  were  ever  delivered  or 
put  into  use  at  DLI.  The  only  contract  for  basic  course  devel¬ 
opment  since  this  period  did  not  produce  satisfactory  results 
either.  Failure  of  the  1969-75  contracts  has  been  attributed 
primarily  to  poor  contract  specifications. 

DLI  has  entered  into  only  one  contract  for  basic  course 
development  since  the  1969-75  period;  this  was  for  the  German 
Basic  Course.  The  contract  was  awarded  in  September  1980  for  the 
amount  of  $25,460  and  was  terminated  in  May  1981.  Although  all 
the  lessons  specified  in  the  initial  contract  were  received,  the 
materials  were  not  usable.  According  to  DLI  officials,  specifi¬ 
cations  were  not  at  fault  for  failure  of  the  contract.  Instead, 
they  said,  DLI ' s  inability  to  effectively  monitor  and  control 
the  contract  caused  its  failure. 

CONCLUSIONS 


Existing  basic  courses  have  deteriorated  to  the  point  where 
there  is  a  detrimental  effect  on  the  quality  of  training.  Course 
development  projects  have  proceeded  slowly  because  of  the  lack  of 
appropriate  priorities,  improperly  implemented  course  development 
procedures,  ineffective  monitoring  and  management  of  development 
projects,  failure  to  fully  explore  the  potential  benefits  of 
commercial  texts,  and  unsuccessful  contracting  efforts. 

DLI  needs  to  develop  a  system  based  on  internal  as  well  as 
external  inputs  for  assessing  course  development  priority  needs. 
Establishing  controls  over  its  course  development  projects  along 
with  an  evaluation  of  alternative  course  materials  should  improve 
course  development. 

RECOMMENDATIONS 


We  recommend  that  the  Secretary  of  the  Army  direct  the  Com¬ 
mandant  of  DLI  to 

— develop  resident  basic  courses  using  commercially  avail¬ 
able  materials  whenever  these  can  be  adapted  at  less  cost 
and  in  less  time  than  in-house  development  and 

— establish  controls  over  course  development  projects  which 
provide  the  means  to  assess  progress  against  specified 
target  dates. 
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AGENCY  COMMENTS 


In  their  March  19,  1982,  comments  (see  app.  I),  DOD  officials 
agreed  that  DLI  could  have  better  managed  its  course  development 
resources;  however,  they  emphasized  that  their  investment  in 
course  development  would  begin  to  be  realized  in  1982  with  the 
completion  of  the  Basic  Russian  course.  We  have  not  verified 
that  DOD  will  meet  the  projected  completion  date  for  the  Basic 
Russian  course.  In  addition,  DOD  in  March  1982  reported  the  sta¬ 
tus  for  several  additional  basic  language  courses  but  did  not 
provide  any  estimated  completion  dates  for  these  courses,  and  we 
have  not  attempted  to  verify  the  provided  information.  While  DOD 
commented  that  it  had  used  and  adapted  commercial  materials  for 
several  of  its  courses,  we  found  only  very  limited  use  of  commer¬ 
cial  materials  and  continue  to  believe  that  greater  use  is  neces¬ 
sary  if  the  Institute  is  to  achieve  its  course  development  goals. 
DOD  officials  reported  that  production  control  measures  had  been 
recently  instituted  to  more  closely  monitor  the  progress  of  course 
development  activities. 

DOD  comments  indicated  that  the  5-year  training  development 
plan  establishes  project  priorities,  and  the  Institute  and  user 
agencies  now  agree  on  resident  and  nonresident  course  development 
priorities.  DOD  specifically  commented  that,  as  of  March  1982, 

74  percent  of  course  development  resources  have  been  allocated  for 
resident  programs  while  26  percent  have  been  allocated  for  devel¬ 
opment  of  nonresident  and  refresher/maintenance  programs.  Accord¬ 
ingly,  we  have  dropped  our  proposed  recommendation  to  establish 
a  more  effective  process  for  setting  project  priorities. 

DOD  commented  that  it  had  acconplished  our  proposed  recommen¬ 
dation  to  establish  controls  over  course  development  activities 
by  late  1981.  However,  the  recency  of  DOD's  actions  and  the  lack 
nf  information  as  to  how  these  actions  will  achieve  the  intent  of 
our  proposed  recommendation  cause  us  to  continue  to  believe  that 
controls  are  needed. 
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CHAPTER  3 


CLASSROOM  INSTRUCTION  CAN  BE  BETTER 
MANAGED  AND  SUPERVISED 


Important  elements  of  classroom  instruction  could  be  better 
managed  and  supervised.  Many  problems  have  been  identified  in 
past  studies  conducted  by  DLI  and  other  organizations.  However, 
to  date,  some  important  training  policies  and  procedures  are 
either  lacking  or  unclear  or  have  been  poorly  implemented.  We 
found  that! 


— DLI  lacked  an  officially  approved  and  accepted  teaching 
methodology  for  instructors  to  use. 

— Instructors,  once  hired,  received  only  limited  training 
in  classroom  instruction. 

— Instructors  were  not  being  adequately  evaluated  on  their 
instructional  capabilities. 

— DLI's  evaluations  of  training  quality  were  not  effective 
because  of  poor  response  to  recommendations  and  inadequate 
followup  on  them. 

— Technical  language  assistants  (TLAs)  provided  to  DLI  have 
not  been  used  effectively. 

While  these  problems  have  not  been  solved,  recent  DLI  initiatives, 
such  as  revising  instructor-hiring  standards,  instituting  a  new 
program  to  improve  the  use  of  TLAs,  and  creating  and  filling  the 
position  of  Academic  Dean,  are  all  aimed  at  improving  classroom 
instruction. 

OFFICIALLY  APPROVED  AND  ACCEPTED 
TEACHING  METHODOLOGY  IS  LACKING 


DLI  has  had  no  definitive  policy  on  methodology  or  the  theory 
of  foreign  language  training  since  January  1976.  Before  1976,  a 
definitive  "official  policy"  on  methodology  was  contained  in  DLI 
Pamphlet  350-4,  entitled  "DLI  Guidelines."  This  pamphlet  dealt 
with  the  principles  and  methods  of  teaching  and  learning  in  the 
Defense  Language  Program.  However,  in  January  1976  the  pamphlet 
was  rescinded  and  has  not  been  replaced.  Although  two  memorandums 
dealing  with  course  methodology  were  written  after  the  pamphlet 
was  rescinded,  they  were  not  adopted  internally  by  DLI  as  official 
guidance  to  the  departments. 

Teaching  methodology  at  DLI  varies  widely  even  within  the 
same  language  department.  Interviews  with  department  heads, 
supervisors,  instructors,  and  students  substantiated  the  use 
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of  different  methodologies.  They  variously  described  the 
methodologies  as  "eclectic,  audio-lingual,  cognitive,  inductive, 
pluralistic,  improvisational ,  and  doing  their  own  thing."  One 
instructor  claimed  that  six  different  methodologies  were  used 
indiscriminately  in  his  department.  Other  instructors  said  they 
had  no  official  DLI  methodology,  or  they  simply  followed  the  text¬ 
book.  DLI  officials  acknowledged  that  the  use  of  various  method¬ 
ologies  had  an  unpredictable  effect  on  the  quality  of  instruction 
and  that  language  departments  had,  in  effect,  been  allowed  to  do 
"their  own  thing."  They  also  acknowledged  that  DLI  needed  to  de¬ 
velop  and  "package"  a  methodology  to  make  it  easier  to  understand 
and  follow. 

INSTRUCTOR  TRAINING  IS  LIMITED 

Many  of  the  newly  hired  instructors  are  not  completing  basic 
instructor  training,  and  even  fewer  receive  additional  training 
for  self-development  and  job  advancement.  Although  DLI  gives  most 
new  instructors  some  training  and  orientation,  it  haB  not  been  con¬ 
sistent  in  routinely  updating  and  reinforcing  the  earlier  training 
with  additional  training. 

Training  records  showed  that  77  percent  of  instructors  hired 
between  January  1980  and  May  1981  received  "Basic  Instructor 
Training,  Phase  I."  This  2-week  course,  supplemented  by  a  1-week 
in-class  observation,  is  designed  to  give  native  or  near-native 
speakers  of  foreign  languages  the  skills,  knowledge,  and  abili¬ 
ties  to  function  as  DLI  instructors.  "Basic  Instructor  Training, 
Phase  II,"  is  an  observation  period  during  which  an  instructor  is 
evaluated  on  how  well  he  or  she  applies  the  techniques  learned 
in  phase  I.  During  the  same  period,  however,  only  16  percent  of 
those  who  completed  phase  I  completed  phase  II.  Further,  records 
indicated  that,  during  this  period,  few  instructors  attended  other 
DLI  courses. 

SUPERVISORS  NOT  PROPERLY  EVALUATING 
INSTRUCTORS 

Language  department  supervisors  are  not  properly  evaluating 
instructors'  classroom  performance.  DLI  guidelines  specify  the 
most  important  tool  in  performance  evaluation  is  the  supervisor 
audit.  This  is  an  unscheduled  visit  to  a  class  by  a  department 
supervisor  for  observing  and  recording  on-the-job  behavior  and 
appraising  performance.  Each  audit  should  include  such  events 
as  observing  behavior,  writing  observations,  discussing  perform¬ 
ance  with  the  instructor,  counseling  the  instructor  as  required, 
and  insuring  the  appropriate  observation  form  is  cosigned  by  the 
instructor. 

The  supervisor  is  responsible  for  observing  instructor 
performance  for  a  full  teaching  period  at  a  minimum  of  6  times 
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a  year  or  more  often  if  necessary.  Instructor  observation  forms 
serve  as  records  of  an  instructor's  performance  and  as  support 
for  written  performance  appraisals. 

While,  in  theory,  the  supervisor  visit  is  an  important  evalu¬ 
ation  tool,  we  found  that  supervisors  were  not  carrying  out  these 
responsibilities.  For  example,  we  interviewed  14  of  the  28  super¬ 
visors  (or  50  percent)  in  the  6  largest  departments,  all  of  whom 
indicated  that  they  did  not  visit  their  instructors'  classes  reg¬ 
ularly;  about  half  indicated  they  visited  classes  as  infrequently 
as  twice  a  year.  Furthermore,  they  did  not  always  record  their 
observations.  Also,  instructors  in  four  of  these  departments  said 
they  had  not  been  counseled  by  supervisors,  although  this  is  re¬ 
quired  after  the  visit. 

DLI  officials  said  they  were  developing  a  new  performance 
appraisal  policy  and  related  procedures  which  were  expected  to 
provide  additional  guidance  on  instructor  evaluations. 

RESPONSE  TO  TRAINING  RECOMMENDATIONS 
HAS  BEEN  POOR  AND  FOLLOWUP  HAS  BEEN 
INEFFECTIVE 

Reviews  of  DLI's  instructional  delivery  system.  Instructional 
Systems  Audits  (ISAs),  are  performed  to  determine  the  means  of 
improving  the  effectiveness  or  efficiency  of  classroom  instruction. 
However,  procedures  for  monitoring  and  implementing  ISA  recommen¬ 
dations  were  not  followed.  Specifically,  we  found  that; 

— The  Directorate  of  Evaluation  had  never  implemented  moni¬ 
toring  procedures  for  ISA  recommendations  although  a 
September  1,  1978,  DLI  memorandum  indicated  such  procedures 
existed. 

— The  Directorate  of  Training  had  delegated  to  its  individual 
language  departments  the  responsibility  for  implementing 
ISA  recommendations  and  had  not  insured  compliance. 

The  Director  of  Training  acknowledged  that  implementing  ISA 
recommendations  had  been  left  to  the  departments.  The  Director 
said  he  had  made  a  conscious  attempt  to  decentralize  authority 
and  thereby  allowed  the  departments  greater  control. 

This  delegation  of  authority,  however,  apparently  did  not 
result  in  timely  training  improvements.  In  a  memorandum  to  the 
Director  of  Training  in  September  1980,  the  former  Commandant  at 
DLI  noted  that  one  department's  reply  was  so  general  that  it  led 
him  to  believe  the  department  took  the  ISA  report  under  advise¬ 
ment  rather  than  identify  the  tasks  needed  to  be  accomplished  in 
order  to  enhance  the  operation  of  the  department  and  that,  had  a 
status  report  not  been  requested,  the  recommendations  would  not 
have  been  seriously  followed  up. 
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In  an  apparent  effort  to  improve  department  responsiveness 
to  ISA  recommendations,  a  revision  to  DLI  Memo  350-5,  dated 
January  15,  1981,  assigned  monitoring  of  ISA  recommendations 
to  the  newly  formed  Office  of  the  Academic  Dean.  However,  to 
date,  written  procedures  for  carrying  out  this  responsibility 
have  not  been  developed.  DLI  officials  told  us,  though,  that, 
as  of  September  1981,  new  ISA  procedures  were  being  readied 
for  dissemination. 

TLAs  HAVE  NOT  BEEN  USED  EFFECTIVELY 


TLAs  have  not  been  effectively  used,  and  no  central  authority 
has  coordinated  their  use  by  individual  departments.  About  60 
TLAs  have  been  assigned  to  language  departments  and  course  develop¬ 
ment.  These  were  career  military  linguists  with  field  experience 
who  could  give  students  a  practical  view  of  the  application  of 
foreign  language  training  to  actual  job  duties.  Their  duties  at 
DLI  included,  but  were  not  limited  to: 

— Explaining  military  terminology. 

— Assisting  in  conducting  and  grading  language  laboratory 
work. 

— Assisting  faculty  in  classroom  instruction,  administering 
tests,  supervising  of  study  halls,  and  tutoring. 

During  the  Special  Program  Review  in  December  1980  and  the 
Annual  Program  Review  in  February  1981,  disagreement  arose  be¬ 
tween  DLI  and  NSA  regarding  the  TLAs'  role.  NSA  contended  that 
DLI  intended  to  use  the  TLAs  as  counselors  rather  than  to  assist 
in  mission  accomplishment  as  originally  intended.  It  further 
contended  that  this  shift  had  a  "deleterious  effect  upon  both 
TLA  morale  and  effectiveness"  and  that  the  TLAs'  language  exper¬ 
tise  could  be  better  used  elsewhere  in  the  defense  community. 
Although  several  of  the  departments  we  reviewed  had  assigned  mean¬ 
ingful  duties  to  TLAs,  others  had  not.  One  department  considered 
the  TLAs  to  be  "spies"  for  the  user  agencies  and  refused  to  allow 
them  significant  roles  in  the  instructional  program. 

DLI  officials  acknowledged  that  some  departments  did  not  use 
TLAs  effectively.  However,  they  pointed  out  that,  in  response  to 
user  agency  criticisms  and  as  an  attempt  to  correct  shortcomings 
in  the  previous  memorandum  of  understanding,  a  new  program  had  re¬ 
cently  been  established.  This  program,  implemented  on  August  28, 
1981,  is  intended  to  insure  proper  use  of  the  TLAs  at  DLI  as  well 
as  to  develop  a  "cadre  of  expert  military  linguists."  Overall, 
the  program  assigns  to  the  Commandant  operational  control  over  all 
TLAs  and  assigns  program  responsibility  to  the  Director  of  Training. 
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CONCLUSIONS 


DLI  officials  must  exercise  greater  oversight  over  the 
school's  instructional  system  so  as  to  insure  an  optimum  level 
of  instructional  quality.  Lack  of  official  policy  guidance  on 
training  methodology,  instructor  training,  and  instructor 
evaluations  and  inadequate  or  untimely  response  to  suggested 
improvements  to  the  instructional  system  are  degrading  the  qual¬ 
ity  of  language  training  linguists  receive  and  could  adversely 
affect  their  job  performance. 

RECOMMENDATIONS 


We  recommend  that  the  Secretary  of  the  Army  require  the  Com¬ 
mandant  to: 

— Develop  and  distribute  a  standard  schoolwide  training 
methodology  for  use  in  all  DLI 1 s  language  departments. 

— Require  all  newly  hired  instructors  to  complete  both  phases 
of  the  basic  instructor-training  course.  Instructors  should 
be  encouraged  to  seek  out  additional  training  to  improve 
their  instructional  abilities  as  part  of  the  individual  de¬ 
velopment  programs. 

— Establish  procedures  to  carry  out  the  reinstated  policy  for 
supervisory  classroom  visits  and  hold  supervisors  accountable 
for  routinely  observing  instructor  classroom  behavior. 

AGENCY  COMMENTS 


DOD  did  not  directly  address  our  recommendations  to  develop 
and  distribute  a  standard  training  methodology  for  use  in  all 
language  departments.  It  responded  that  DLI  had  begun  a  major 
revision  to  its  entire  faculty  professional  development  program. 
DOD  reported  that,  from  January  1980  to  February  22,  1982,  a  total 
of  506  faculty  members  received  additional  training  aside  from 
the  Basic  Instructor  Training  Workshops.  As  noted  on  page  12,  we 
addressed  only  newly  hired  instructors  through  May  1981  and  pri¬ 
marily  the  second  phase  of  basic  instructor  training  rather  than 
additional  training  reported  in  DOD  comments  for  its  faculty. 

Revision  of  DLI  faculty  professional  development  program, 
realignment  of  the  Faculty  and  Staff  Development  Division  under 
the  Office  of  the  Academic  Dean,  and  changes  to  procedures  for 
responding  to  the  results  of  Instructional  System  Audits  and 
field  evaluations  were  reported  as  recent  measures  which  should 
help  to  improve  the  management  of  DLI  classroom  training. 
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Concerning  the  proposed  recommendation  in  our  draft  report 
for  requiring  additional  training  for  new  instructors,  DOD  offi¬ 
cials  responded  that  the  requirement  for  additional  training  for 
new  instructors  could  best  be  incorporated  in  individual  develop¬ 
ment  plans.  We  concurred  in  this  approach  for  managing  instructor 
training  and  have  modified  the  recommendation  accordingly.  (DOD's 
official  comments  are  included  as  app.  I.) 

The  intent  of  our  proposed  recommendation  on  supervisory 
classroom  visits  was  to  establish  a  policy  and  implement  it.  DOD 
commented  that  the  policy  for  supervisory  visits  to  classrooms 
which  had  been  rescinded  in  the  1970s  was  recently  reinstated.  It 
did  not  comment  on  how  the  policy  is  to  be  implemented  or  if  and 
how  supervisors  will  be  held  accountable  for  following  the  policy. 
We  have  therefore  revised  our  proposed  recommendation  to  provide 
for  a  management  control  that  can  be  used  to  insure  compliance 
with  the  supervisory  visits  policy. 
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CHAPTER  4 


DLI  LACKS  AN  ADEQUATE  MECHANISM  FOR 
DETERMINING  THE  QUALITY  OF  ITS  STUDENTS 

DLI ' s  existing  evaluation  process  is  inadequate  for 
assessing  student  proficiency  or  determining  how  well  the  DLI 
is  performing  its  language-training  mission.  Specifically,  we 
found  that: 

— DLI  did  not  have  a  cohesive  statement  of  its  training 
objectives  and  standards. 

— Proficiency  testing  had  not  been  adequately  developed  as 
an  evaluation  tool  and  the  Defense  Language  Proficiency 
Test  ( DLPT )  was  not  relied  on  for  determining  proficiency 
of  graduates. 

— Development  of  criterion-referenced  tests,  which  measure 
achievement  of  users'  terminal  learning  objectives  (TLOs), 
had  not  progressed. 

DLI  DOES  NOT  HAVE  A  COHESIVE  STATEMENT 
OF  ITS  TRAINING  OBJECTIVES  AND  STANDARDS 


Before  any  evaluation  of  DLI ' s  training  system  can  be  effec¬ 
tive,  there  must  be  a  clear  understanding  of  exactly  what  the 
training  objectives  and  standards  are.  Yet  DLI ' s  objectives 
and  standards  are  not  clear,  and  this  has  caused  confusion  over 
what  to  evaluate  or  what  the  proficiency  level  of  graduates 
should  be. 

Since  1976  DLI  has  not  had  a  single  cohesive  policy  docu¬ 
ment  clearly  explaining  the  training  objectives  and  standards 
and  their  interrelationship.  Before  this  time,  however,  DLI's 
training  objective  was  to  give  students  a  foundation  in  the 
language  sufficient  to  attain  proficiency  level  2  (limited  work¬ 
ing  proficiency)  in  reading  and  writing  and  level  3  (minimum 
professional  proficiency)  in  listening  and  speaking,  but  such 
general  language  training  was  not  designed  to  prepare  students 
for  any  particular  type  of  mission  or  assignment. 

As  described  by  FSI's  proficiency  index,  one  who  has  achieved 
level  2  for  reading  can  read  simple  narrative  on  familiar  subject 
matter  and,  aided  by  a  dictionary,  he/she  can  get  the  general 
sense  of  written  communications.  One  who  has  achieved  level  2  for 
writing  can  write  sentences  on  familiar  topics  appropriately 
using  technical  language  vocabulary;  errors  in  spelling  and  struc¬ 
ture  occasionally  obscure  the  meaning  of  written  material.  On 
the  other  hand,  level  3  listening  and  speaking  require  more  abil¬ 
ity.  To  achieve  level  3  listening,  an  individual  should  be  able 
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to  understand  general  conversation  or  discussion  on  topics  within 
his/her  special  field.  Similarly,  a  person  speaking  at  level  3 
must  be  able  to  participate  effectively  in  all  general  conversation 
and  discuss  particular  interests  in  his/her  special  field  without 
making  errors  that  obscure  meaning.  (App.  IV  lists  FSI's  proficiency 
level  descriptions.) 

In  1977,  DLI  requested  and  received  from  its  user  agencies 
a  new  set  of  TLOs  which  more  clearly  specified  the  skills  that 
students  should  master  to  be  able  to  perform  their  language  duties. 
The  TLOs  consist  of  25  objectives  which,  if  achieved,  would  yield 
a  basic  language  student  able  to  assume  assigned  linguistic  tasks. 
Examples  of  TLOs  involve  such  objectives  as  spoken  interpretation, 
reading  interpretation,  conversational  response,  transcription/ 
written  response,  etc.  However,  the  TLOs  when  received  did  not 
specify  quantitative  standards  by  which  to  measure  achievement  of 
these  objectives.  Such  standards  should  spell  out  the  performance 
level  that  would  be  acceptable  to  users.  For  example,  reading 
interpretation  is  one  skill  identified  in  the  TLOs.  To  satis¬ 
factorily  demonstrate  attainment  of  this  skill,  standards  should 
specify  the  level  of  accuracy  that  would  be  generally  acceptable; 
that  is,  the  individual  should  accurately  interpret  all  information 
conveyed  or  75  percent  of  the  information  and/or  must  be  able  to 
organize  it  in  the  sequence  originally  conveyed. 

Notwithstanding  the  lack  of  specific  standards,  DLI,  in  1977, 
began  to  incorporate  the  new  TLOs  into  its  policies  and  training 
program.  However,  without  specific  standards,  DLI  did  not  know 
whether  the  TLOs  were  being  achieved,  and  DLI  continued  to  provide 
instruction  and  graduate  students  on  essentially  the  same  basis 
that  it  had  done  before  receiving  the  new  TLOs. 

Because  of  the  absence  of  specific  standards,  various  DLI 
staff  with  whom  we  spoke  had  differing  understandings  of  what 
DLI 1 s  mission  and  course  objectives  were.  Some  said  DLI 1 s  basic 
courses  were  supposed  to  meet  proficiency  level  3,  others  told 
us  level  2,  and  some  said  the  courses  should  meet  the  TLOs. 

Notably,  there  is  no  recognized  connection  between  the  profi¬ 
ciency  level  descriptions  and  TLOs. 

The  Director  of  Evaluation,  in  a  March  1981  memorandum  to 
the  Commandant,  characterized  the  ambiguity  over  DLI ' s  mission 
objectives  and  training  standards  as  a  "systemic  problem."  He 
explained  that  TLOs  were  only  tangentially  addressed  in  the 
course  objectives,  the  graduation  criteria,  the  instruction, 
or  the  final  examination  and  that  the  ambiguity  of  proficiency 
level  descriptions  resulted  in  inconsistent  interpretations  by 
instructors . 

DLI  officials  with  whom  we  discussed  this  matter,  includ¬ 
ing  the  Director  of  Training  Development  and  the  Director  of 
Evaluation,  agreed  that  ambiguity  in  mission  statement  and  course 
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objectives  and  the  lack  of  training  standards  were  causing 
misunderstanding  over  what  DLI  should  be  expected  to  accomplish 
and  that  cohesive  policy  guidelines  were  needed  similar  to  those 
which  existed  in  1976. 

CURRENT  PROFICIENCY  TESTS  ARE  NOT  RELIED 
ON  AS  A  MEASURE  OF  THE  EFFECTIVENESS  OF 
DLI 1 s  TRAINING  SYSTEM 


The  only  yardstick  for  measuring  the  overall  effectiveness  of 
DLI  training  or  student  proficiency  is  the  DLPT .  Yet,  even  though 
DLI  administers  the  DLPT,  it  is  not  a  graduation  requirement  and 
it  is  not  relied  upon  as  a  primary  measure  of  training  effectiveness. 
DLI  officials  pointed  out  that  the  DLPT  had  been  designed  to  screen 
personnel  for  general  language  proficiency  and  had  never  been  vali¬ 
dated  against  objective  standards  of  proficiency.  Nonetheless,  a 
1973  GAO  report  1/  concluded  that  reliable  proficiency  testing 
was  needed  to  evaluate  training  effectiveness.  The  DLPT  admin¬ 
istered  by  DLI  consists  of  120  multiple-choice  items  covering 
2  (listening  and  reading)  of  the  4  language  skills  taught  at 
DLI  (additionally,  speaking  and  writing)  and  takes  about  1  hour 
to  administer.  Scores  are  converted  into  proficiency  levels 
ranging  from  level  0  (no  proficiency)  to  level  3  (minimum  pro¬ 
fessional)  on  a  scale  developed  years  ago  by  FSI. 

Although  DLI  administers  the  DLPT  at  the  time  of  graduation, 
it  does  not  rely  on  the  scores  because  there  is  no  assurance  of 
how  accurately  proficiency,  as  indicated  by  the  FSI  proficiency 
levels,  equates  to  the  DLPT  because  the  method  of  converting  or 
transforming  DLPT  scores  to  FSI's  proficiency  levels  has  never 
been  validated.  In  other  words,  it  has  never  been  scientifically 
established  as  to  what  raw  scores  from  the  DLPT  equate  to  FSI 
levels.  DLI  officials  acknowledged  that  lack  of  validation 
diminished  confidence  in  the  meaning  of  the  proficiency  levels 
assigned  when  students  completed  the  DLPT.  They  said,  however, 
that  a  project  was  underway  which  would  establish  generally 
accepted  standards  so  that  DLPT  scores  would  have  more  meaning. 

DLI ' s  Director  of  Evaluation  and  the  Chief  of  Test  Division  both 
agreed  that  the  results  of  this  project,  if  successful,  would  be 
a  more  useful  measure  of  proficiency  and  that  it  would  not  be 
unreasonable  to  require  that  students  attain  the  levels  specified 
in  whatever  training  objectives  that  DLI  decided  to  establish. 

Students  of  DLI  graduate  solely  on  the  basis  of  scores  they 
achieve  on  various  interim  tests  and  a  final  test  at  the  end  of 
the  course,  instead  of  proficiency  test  results.  These  tests 
measure  achievement  of  course  contents  and  are  not  related  to 
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the  FSI  proficiency  levels.  Although  a  correlation  exists  between 
end-of-course  grades  and  DLPT  proficiency  levels,  DLI  does  graduate 
students  who  do  not  attain  a  high  level  of  proficiency,  even  though 
some  students  achieve  high  scores  for  their  final  course  grades. 

For  example,  of  about  26,000  students  who  graduated  between  1974 
and  1981.  2,661  graduates  attained  level  1  or  lower  in  the  listen¬ 
ing  skill.  Similarly,  2,354  graduates  attained  level  1  or  lower 
for  the  reading  skill.  Due  to  the  number  of  graduates  involved, 
we  did  not  try  to  determine  why  they  could  not  attain  level  2  or 
higher  for  these  skills.  For  example,  we  do  not  know  the  extent 
to  which  this  may  be  due  to  errors  in  measurement  or  clerical 
recording  errors  and,  thus,  cannot  say  with  any  assurance  whether 
students  have  mastered  language  training  at  DLI. 

CRITERION-REFERENCED  TESTS  HAVE 
BEEN  DEVELOPED  SLOWLY — TESTS  SHOULD 
REFERENCE  TLOs 

DLI’s  course  tests  have  been  the  subject  of  criticism.  In  its 
review  of  DLI  graduates  assigned  to  military  field  units,  DLI 1 s 
Directorate  of  Evaluation  concluded  that  DLI  needed  to  devise  tests 
for  measuring  the  skills  students  had  been  taught  at  Monterey.  DLI 
officials  acknowledged  that  developing  criterion-referenced  tests 
would  overcome  the  deficiencies  in  existing  tests;  however,  develop¬ 
ment  of  these  tests  has  been  ongoing  since  1978,  with  very  little 
progress . 

Criterion-referenced  tests  are  designed  to  measure  how  well 
students  have  learned  language  skills  specified  in  the  TLOs.  We 
agree  that  these  tests  should  provide  a  more  objective  measure¬ 
ment  of  achievement.  However,  we  were  told  that  these  tests  were 
not  being  written  and  validated  directly  to  TLOs.  DLI  officials 
said  they  had  experienced  difficulty  in  developing  these  tests 
because  TLOs  did  not  detail  the  level  of  achievement  needed. 
Therefore,  tests  could  not  measure  how  well  course  objectives, 
based  on  TLOs,  had  been  achieved.  Furthermore,  DLI  officials 
stated  that  NSA  had  declined  a  request  to  validate  DLI  tests  in 
actual  job  environments. 

Although  DLI  labels  the  tests  it  is  developing  as  criterion 
referenced,  we  were  told  that  DLI  had  curtailed  its  attempts  to 
writ*1  tests  based  on  TLOs.  Instead,  we  were  told  that  the  new 
tests  being  developed  merely  represented  achievement  tests  on  new 
basic  course  materials  being  developed.  DLI  officials  claimed 
that,  since  new  courses  were  being  developed  on  the  basis  of  ESA’s 
TLOs,  new  tests  would  be  a  better  indicator  of  whether  students 
had  been  able  to  achieve  stated  objectives. 
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CONCLUSIONS 


Lack  of  clear  and  cohesive  training  objectives  and  standards 
to  measure  language  proficiency  has  prevented  DLI  from  effectively 
appraising  its  training  mission.  Although  we  believe  that  student 
language  proficiency  is  the  best  indicator  for  determining  the 
effectiveness  of  training,  DLI  continues  to  appraise  its  overall 
training  effectiveness  on  the  basis  of  student  grades  and  achieve¬ 
ment  tests. 

In  line  with  our  1973  report,  we  continue  to  believe  that 
valid,  reliable  proficiency  testing  is  a  key  element  of  sound 
evaluation.  Such  tests  would  reveal  which  students  were  well 
qualified  for  graduation  and  could  identify  areas  where  training 
could  be  improved.  Although  DLI  is  developing  new  tests  to  com¬ 
plement  new  courses,  they  may  not  be  fully  satisfactory  for 
determining  the  quality  of  its  training  or  the  skills  of  its 
graduates . 

RECOMMENDATIONS 


We  recommend  that  the  Secretary  of  the  Army  direct  the 
Commandant  of  DLI  to: 

— Establish  realistic  training  objectives  based  on  mission 
requirements  and  use  the  DLPT  to  measure  graduate  students ' 
performance  in  satisfaction  of  these  objectives  and  require 
that  students  pass  the  DLPT  as  a  condition  for  graduation. 

— Develop  a  DLPT  that  will  measure  student  proficiency  of  the 
objectives  and  standards  established  by  DLI. 

AGENCY  COMMENTS 


DOD  agrees  that  establishing  training  objectives  based  on  mis¬ 
sion  requirements  is  the  foundation  needed  before  language  profi¬ 
ciency  can  be  measured.  DOD  reported  that  DLI  is  an  active  partic¬ 
ipant  of  the  Interagency  Language  Roundtable  which  is  reviewing 
U.S.  Government  language  proficiency  standards.  These  standards, 
when  published,  will  be  the  benchmarks  for  DLI  training  objectives. 

Furthermore,  DOD  agreed  with  our  proposed  recommendation  to 
establish  realistic  training  objectives  which  can  be  evaluated  by 
using  the  DLPT.  At  DOD'b  suggestion,  we  modified  our  proposed 
recommendation  to  require  the  Secretary  of  the  Army  to  establish 
realistic  training  objectives  based  on  mission  requirements  and 
to  use  the  DLPT  to  measure  performance.  In  addition,  in  accordance 
with  DOD’s  suggestion,  we  added  as  part  of  this  recommendation  that 
passing  the  DLPT  be  a  requirement  for  graduation  and  deleted  the 
separate  recommendation  which  requested  that  the  DLPT  be  used  as 
a  requirement  for  graduation. 
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DOD  officials  did  not  agree  with  our  proposed  recommendation 
to  resolve  the  problem  of  converting  DLPT  scores  to  FSI  proficiency 
descriptions  but  stated  that  DLI  would  redesign  the  DLPT  to  better 
meet  its  needs  for  assessing  student  proficiency.  We  concur  in 
DOD' s  approach  to  assessing  student  proficiency  and  have  changed 
our  proposed  recommendation  accordingly.  (See  app.  I  for  DOD's 
comments . ) 
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APPENDIX  I 

THE  UNDER  SECRETARY  OF  DEFENSE 


WASHINGTON  0  C  20301 


1  S  MAR  ! 982 


Mr.  Clifford  I.  Gould 
Director,  Federal  Personnel 
and  Compensation  Division 
United  States  General  Accounting 
Of f i ce 

Washington,  D.  C.  20548 
Dear  Mr.  Gould: 

This  is  in  reply  to  your  letter  to  the  Secretary  of  Defense  regarding  your 
report  dated  February  18,  1982  on  "Weaknesses  in  the  Resident  Language 
Training  System  of  Defense  Language  Institute  Affect  the  Quality  of  Trained 
Linguists"  OSD  Case  #5904,  GAO  Code  961149.  As  enunciated  in  the  March  5, 

1982  meeting  between  Messrs.  Kremer  and  Esposito  from  GAO,  and  personnel 
from  the  Department  of  Army  Staff  and  the  Office  of  the  Secretary  of  Defense, 
we  share  a  common  Interest  in  the  effectiveness  of  training  at  the  DLIFLC. 

We  also  feel  that  there  has  been  a  high  level  of  accomplishment  at  the  DLI, 
and  that  many  problems  identified  in  the  past,  actions  taken,  programs 
developed,  and  resources  committed  are  producing  desirable  results.  Our 
Investment  In  the  DLI,  like  any  other  investment  strategy,  includes  a 
commitment  of  resources--funds,  personnel,  and  management  attention.  But 
it  also  Includes  time,  time  for  the  Investment  to  realize  its  dividend. 

Many  of  the  observations  of  the  GAO  review  team  were  also  made  by  DoD 
personnel,  some,  years  ago.  The  effects  of  many  of  our  corrective  actions 
have,  in  fact,  been  realized  since  the  departure  of  the  on-site  GAO  team 
last  September  (1981).  Therefore,  we  think  it  appropriate  to  offer  as  detailed 
an  update  as  possible  to  the  draft  report  In  order  for  the  GAO  to  present  the 
most  accurate  picture  of  this  important  program  to  the  Congress. 

As  agreed  to  during  the  March  5th  meeting,  our  comments  are  divided  into  two 
major  parts  and  are  attached: 

-  Enclosure  1 :  proposed  additional  paragraph  to  Cover  Summary;  comments 
on  Digest,  recommendations,  and  proposed  "Agency  Comments"  paragraph 

-  Enclosure  2:  a  by-chapter  update. 

We  hope  these  comments  can  be  used  to  enhance  the  report,  and  to  picture  the 
DLI  in  light  of  current  efforts.  Action  In  response  to  final  GAO  recommenda¬ 
tions  will  be  accomplished  by  the  Office  of  the  Secretary  of  Defense  and  the 
Secretary  of  the  Army  as  appropriate. 
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Recommended  additional  paragraph  to 
"Cover  Summary" 

GAO  acknowledges  that  many  of  the  issues  and  problems  cited  in  this  study 
have  also  been  recognized  by  the  Department  of  Defense,  and  that  corrective 
action  has  been  started  and  in  some  cases  accomplished.  The  increase  in  resources 
committed  to  support  the  Institute,  the  level  of  management  attention  devoted  by 
Army  and  DoD-wide  users,  and  recent  internal  organizational  and  leadership  changes 
are  indicative  of  movement  toward  an  improved  training  effort. 

Comments  on  "DIGEST" 


Page  i :  (1)  The  DLI  has  no  responsibility  for  foreign  language  training  at  the 

k  Service  Academies  or  in  Overseas  Dependent  Schools. 

(2)  Although  changes  in  the  "management  of  training"  at  the  DLI  may 
help  to  "improve  the  quality  of  language  trained  personnel,"  it 
is  necessary  to  note  that  the  training  enterprise  is  only  one 

>  subset  of  a  much  larger  group  of  human  resource  considerations 

i  affecting  personnel  quality— to  include  a  broad  range  of  personnel 

administrative  concerns,  manpower  management,  compensation, 
recruitable  labor  sources,  uniqueness  of  mission,  deployment, 
and  utilization,  (indue  weight  on  management  changes  at  DLI  to 
change  the  character  of  Defense  human  resources  may  be  misleading. 
The  DoD  is,  in  fact,  attempting  to  improve  linguist  personnel  by 
addressing  a  much  broader  set  of  manpower  and  personnel  issues. 

Page  ii :  (1)  The  investment  of  resources  (dollars  and  manyears)  for  training 

development  has  not  been  without  yield.  The  investment  started 
less  than  four  years  ago  and  new  courses  will  be  completed 
beginning  calendar  1982  with  continuous  completions  each  year 

>  thereafter.  A  production  time  is  a  necessary  and  understandable 

component  of  any  investment/development  program.  An  accurate 
accounting  of  curriculum  developments  is  offered  in  the  comments 
on  Chapter  2. 

(2)  Lack  of  rapid  progress  in  course  development  is  also  attributable 
to  a  deliberate  management  decision  to  use  limited  resources  to 
meet  the  priority  mission— resident  training  of  a  rapidly 
increasing  student  load. 

Comments  on  "Recommendations" 

Recommendation  #1  -  In  addition  to  commercial  sources,  DLI  has  also  used  other 
government  agency  and  university  programs. 

Recommendation  #3  -  Already  accomplished 
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Recommendation  #5  -  Recommend  second  sentence  to  read,  "...  to  seek  out 
additional  training  to  improve  instructional  capabilities  as  part  of 
Individual  Development  Programs." 

Recommendation  #6  -  A  policy  of  supervisory  visits  to  classrooms  exists.  It 
had  been  temporarily  recinded  but  has  been  reinstated. 

Recommendation  #7  -  Training  objectives  are  not  derived  through  use  of  proficiency 
tests.  Recommend  rewrite  to  read: 

"Establish  realistic  training  objectives  based  on  mission 
requirements.  Use  the  Defense  Language  Proficiency  Test  to 
measure  student  performance  in  satisfaction  of  those  objectives." 

Recommendation  #3  -  Nonconcur.  DoD  has  no  intention  of  converting  DIPT  scores 
to  FSI  proficiency  levels.  Current  work  in  redesigning  DLPT's  is  being 
coordinated  with  the  State  Department  and  other  interested  agencies. 

Recommendation  § 9  -  Recommend  merge  with  recommendation  #7,  they  are  almost 
the  same  thing. 

Recommended  Additional  Paragraph 

Agency  Comments: 

The  Department  of  Defense  shares  a  common  interest  with  the  GAO  in  the 
effectiveness  of  training  at  the  DLIFLC.  We  feel  that  there  has  been  a  high 
level  of  accomplishment  and  that  many  problems  identified  in  the  past,  actions 
taken,  programs  developed,  and  resources  committed  are  producing  desirable 
results.  The  increased  investment  in  the  DLI  should  be  viewed  as  any  other 
investment:  resources  plus  time  yields  a  dividend.  Many  of  the  observations 
of  the  GAO  review  team  were  also  made  by  DoD  personnel  as  long  as  several  years 
ago.  The  Impact  of  many  of  our  corrective  actions  is  just  now  being  felt, 
even  though  some  were  instituted  years  ago. 

The  Defense  Foreign  Language  Training  Program,  and  the  Institute  in 
Monterey  are  high  priorities  for  the  Department  of  the  Army  and  the  entire 
Defense  community.  We  believe  it  is  well  on  its  way  to  accomplish  mission 
requirements  in  a  most  effective  and  efficient  way. 


25 


APPENDIX  I 


APPENDIX  I 


The  following  comments  are  offered  to  provide  a  more  complete  picture  of  DLIFLC 
operations.  Some  of  these  comments  refer  to  past  actions,  some  to  recent  DLIFLC 
in-house  initatlves,  and  some  to  new  cooperative  projects  with  other  government 
agencies.  Details  on  many  of  these  programs  can  be  found  in  the  February  1982 
GOSC  and  APR  reports.  The  lauditory  comments  and  spirit  of  cooperation  evidenced 
by  all  in  attendance  at  these  conferences  were  in  and  of  themselves  a  testimony 
of  DLIFLC  progress  across  a  whole  system  of  training  activities. 


Chapter  1 

Page  2 :  (1)  The  DLI  provides  training  in  37  major  languages  and  dialects. 

(2)  The  DLI  determines  course  lengths  in  coordination  with  the  Services 
and  user  agencies. 

(3)  The  Commandant  maintains  administrative  control  of  all  Army  students 
at  the  DLI. 


Page  3:  The  1980  GAO  report  on  foreign  language  needs  neither  evaluated, 
analyzed,  nor  reported  on  extensive  data  provided  by  the  Department  of 
Defense  concerning  Defense  foreign  language  requirements  and  capabilities. 
The  Defense  information  was  provided  en  toto  to  th°  interested  Congressional 
Committees  at  DoD  request  and  was  only  used  by  GAO  to  determine  aggregate 
totals.  It  is  completely  misleading  and  false  to  reference  the  1980  report 
as  shedding  any  light  on  DoD  linguistic  competence  or  on  the  value  of 
training  programs. 


Page  4:  We  have  no  record  that  the  1979  and  1981  DLI  external  field  evaluations 
were  generally  accepted  by  Service  officials. 


Chapter  2 


In  the  area  of  course  development,  resident  course  development  has  and  will 
continue  to  take  precedence  over  nonresident  course  development.  However, 
nonresident  projects  will  continue  to  be  supported  because  they  are  needed 
to  refresh,  maintain,  and  improve  the  hand-won  skills  acquired  in  resident 
training  courses.  In  short,  they  are  needed  to  protect  our  linguistic  invest¬ 
ment.  As  of  March  1982,  74  percent  of  course  development  resources  are  allocated 
for  resident  programs;  26  percent  for  nonresident  and  refresher/maintenance 
programs . 


DLIFLC's  priorities  for  course  development  are  set  by  DoD  user  agencies  (NSA 
and  the  Services).  The  current  system  for  establishing  project  priorities 
through  the  TDFYP  is  efficient  and  effective  and  satisfies  the  needs  of  DoD 
user  agencies.  The  TDFYP  was  again  confirmed  by  all  DoD  user  agencies  during 
the  APR  of  Feb  82.  Other  user  agencies  outside  DoD  also  expressed  confidence 
in  the  current  system.  Principals  clearly  confirmed  that  resident  programs 
have  priority  over  nonresident  programs,  that  signal  intelligence  requirements 
have  priority  over  other  needs,  and  that  potential  "threat"  languages  have 
priority  over  "non  threat"  language.  DLIFLC  is  meeting  the  needs  of  DoD  user 
agencies  In  response  to  stated  and  approved  priorities  and  has  recently 
Initiated  a  series  of  production  control  measures  to  Improve  course  development 
programs . 


26 


APPENDIX  I 


APPENDIX  I 


DLIFLC  is  currently  developing  resident  basic  courses  in  Russian  (will  be 
completed  in  Sep  82,  a  validation  edition  is  presently  being  taught  in  the 
classroom),  Korean,  (55%  completed),  German  (10%  completed),  Arabic  (Modern 
Standard),  and  three  dialects;  Egyptian  (85%  completed),  Syrian  (85%  completed); 
and  Spanish  is  also  under  development.  Analysis  and  design  packages  for  all 
Basic  Courses  presently  under  development  have  been  completed.  Analysis  and 
design  are  also  being  initiated  for  new  Italian,  Greek,  and  Japanese  Basic 
Courses.  The  above  listed  programs  were  approved  and  funded  in  the  TDFYP  and 
represent  all  high  density  "threat"  languages.  In  addition,  DLIFLC  is  contin¬ 
uously  updating  all  Basic  Courses  through  an  established  course  maintenance 
program  within  the  individual  departments  under  the  overall  supervision  of  a 
newly  developed  position  of  language  maintenance  coordinator. 

In  concert  with  our  course  development  efforts  DLIFLC  has  not  only  adopted 
an  instructional  systems  development  approval,  but  has  also  contributed  to  the 
state  of  the  art  in  foreign  language  curriculum  development.  DLIFLC  also 
established,  in  late  1981,  a  program  management  which  plans  for  and  obtains 
adequate  resources,  and  monitors  and  assesses  the  progress  of  all  development 
projects.  The  Commandant  and  Academic  Dean  are  provided  a  quarterly  briefing 
on  the  status  of  each  workplan. 

While  it  is  true  that  DLIFLC  could  have  better  managed  course  development 
resources  in  the  past,  performance  In  this  area  should  be  viewed  with  an 
understanding  of  the  massive  effort  associated  with  the  development  of  a 
single  new  Basic  Course.  Our  new  Basic  Courses  provide  In  one  week  the  number 
of  instructional  hours  In  a  quarter  length  college  course.  Furthermore, 
development  goes  far  beyond  just  the  preparation  of  classroom  teaching  materials 
to  tests,  instructor  handbooks,  home  materials,  etc.  Multiply  these  major  efforts 
times  the  number  of  high  density  "threat"  languages  and  one  can  better  understand 
the  nature  of  manpower  expenditures  involved  In  the  DLIFLC  course  development 
program. 

DLIFLC  has  a  long-standing  policy  of  reviewing  commercially  available  text 
materials  for  possible  use  In  its  curricula  and  has  used  and  adapted  commercial 
materials  for  several  courses  from  time  to  time.  The  limited  scope  of  these 
materials,  copyright  restrictions,  frequently  exorbitant  prices,  and  uncertain 
availabilities  are  but  a  few  of  the  factors  which  resulted  in  a  conscious 
decision  to  "develop"  rather  than  "buy"  course  materials  In  the  past.  Not¬ 
withstanding,  these  difficulties  and  uncertainltles,  DLIFLC  still  pursues  the 
acquisition  of  appropriate  commercial  materials  where  appropriate.  It  should 
also  be  noted  that  CIA,  NSA  and  FSI  language  schools  also  rely  primarily  on 
government-produced  materials  In  their  own  high  density  language  program  basic 
courses. 

Similarly,  because  non-government  agencies  have  not  had  experience  developing 
courses  of  the  magnitude  required  for  Intensive  DLIFLC  training,  we  have  learned 
to  proceed  with  caution  in  contracting  course  development  projects.  Of  course, 
where  the  required  expertise  exists,  we  will  continue  to  maintain  the  option 
of  developing  materials  under  contract  as  we  are  now  doing  with  several  test 
development  projects. 
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Chapter  3 

OLIFLC  regards  professional  faculty  development  as  the  cornerstone  of  DLIFLC 
training.  We  have  been  doing  a  lot  in  this  area  and  will  be  doing  even  more. 

Although  the  GAO  report  credits  only  "a  few  instructors"  as  having  completed 
faculty  training  aside  from  the  B1TVI  Phase  1,  actually  506  faculty  members 
received  training  during  the  period  Jan  80  to  22  Feb  82;  training  subject  matter 


and  number  of  faculty  in  attendance  are  set 

forth  below: 

Course  Description 

Jan-May  81 

Jun  81 -Feb  81 

Criterion  Referenced  Instruction(CRI) 

36 

9 

Criterion  Referenced  Tests  (CRT) 

29 

12 

CRT  for  Managers 

6 

0 

CRT  for  Item  Writers 

3 

0 

English  for  Professional  Development 

33 

34 

Group  Dynamics  &  Leadership 

29 

0 

Introduction  to  Linguistics 

81 

20 

Audio-Visual  Training 

58 

19 

Counseling 

22 

58 

Intro  to  Instructional  Systems  and 
Development  and  Lesson  Design 

Validation  Course 

5 

4 

Applied  Linguistics 

31 

15 

Total 

135“ 

171 

While  the  GAO  reports  only  77  percent  of  newly  hired  instructors  as  receiving 
"Basic  Instructor  Training  -  Phase  I,"  nearly  95  percent  of  newly  hired  instructors 
received  the  two  week  Basic  Instructor  Training  Workshop  (BITW)  Phase  I  during 
the  period  Jan  80  to  present.  The  discrepancy  in  the  two  percentages  appears 
to  be  attributable  to  the  fact  that  only  about  70  percent  of  BITW  Phase  I  graduates 
completed  the  one  week  course  of  instruction.  Introduction  to  Linguistics, 
presented  immediately  following  completion  of  Phase  I  training.  Internal  review 
has  found  that  this  linguistics  course  is  not  appropriate  for  beginner  instructor 
training. 

DLIFLC  has  begun  a  major  revision  of  the  entire  DLIFLC  faculty  professional 
development  program,  based  on  data  identified  during  a  Faculty  and  Staff 
Division  Instructional  Systems  Audit  (ISA)  and  follow-on  DLFLC  Faculty  and 
Staff  Division  Task  Force,  and  the  more  recent  detailed  review  by  the  Academic 
Dean. 

Plans  have  also  been  made  for  a  detailed  review  of  DLIFLC  facility  and  staff 
courses  and  curricula  by  a  team  consisting  of  national -known  visiting  professors 
and  selected  members  of  the  DLIFLC  teaching  and  management  staff  who  are  well 
versed  in  professional  faculty  development.  Extensive  review  and  revision  of 
current  faculty  professional  development  course  materials  will  take  place  in 
the  summer  of  CY  82.  A  thorough  review  and  revision  of  other  faculty  develop¬ 
ment  materials  above  and  beyond  BITW  will  also  be  initiated.  Efforts  in  this 
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regard  will  be  directed  toward  several  major  areas:  basic  teacher  training, 
developing  English  language  proficiency  of  the  faculty;  BITW  reinforcement 
training  one  year  following  initial  hire,  supervisor  and  chairperson  managerial 
training;  refresher  training  for  high  tenure  non-supervisory  instructors,  and 
expansion  of  audio-visual  training  in  response  for  increased  use  of  audio-visual 
technology  in  the  classroom. 

While  teaching  methodology  will  be  part  of  this  total  review,  it  should  be 
mentioned  that  DLIFLC  presently  has  an  eclectic  approach  to  language  learning 
comparable  to  that  used  by  CIA,  Fsi,  and  NSA.  Our  failure  has  obviously  been 
in  not  communicating  to  GAO  and  some  members  of  our  own  faculty  and  staff  that 
this  eclectic  approach  is  indeed  a  methodology.  One  of  the  most  immediate 
difficulties  in  this  regard  is  a  problem  of  nomenclature.  It  is  semantically 
impossible  to  talk  about  "the  eclectic  method." 

Overall  management  of  the  Faculty  and  Staff  Development  Division  has  been 
placed  directly  under  the  Academic  Dean's  office  to  insure  that  full  attention 
is  given  to  this  most  important  aspect  of  DLIFLC's  training  mission. 

DLIFLC  has  revised  post-instructional  Systems  Audits  and  Field  Evaluation 
report  procedures.  These  changes  require  that  the  Directorate  of  Evaluation 
not  only  provide  formal  ISA  and  Field  Evaluation  Reports,  but  also  a  personal 
briefing  to  the  Academic  Dean,  concerned  Directorates,  Group  Chiefs  and  language 
Department  Chairpersons.  Additionally,  the  Directorate  of  Evaluation  also  now 
provides  its  recommendations  to  others  at  the  post-ISA/Field  Evaluation  meetings. 
These  recommendations  are  then  discussed  at  length  until  a  consensus  is  reached 
as  to  which  are  practical  and  feasible.  This  represents  a  change  to  procedures 
in  effect  during  the  period  of  the  GAO  report.  At  that  time,  the  Directorate 
of  Evaluation  did  not  discuss  its  recommendation  with  concerned  parties,  with 
the  result  that  often  they  were  not  well  received  and  in  turn,  not  followed 
through.  The  revised  procedures  have  improved  communications  and  led  to  the 
identification  and  implementation  of  more  reasonable  recommendations.  The 
Academic  Dean  personally  chairs  the  post  ISA/Field  Evaluation  meetings  and 
personally  monitors  execution  of  recommendations  made. 

The  Directorate  of  Training,  acting  under  direction  of  the  DLIFLC  Commandant, 
has  launched  a  formal  Defense  Foreign  Language  Professional  Development  Program 
as  a  vehicle  designed  to  improve  management  of  technical  language  assistants 
in  response  to  DLIFLC  and  DoD  user  agency  needs. 

Chapter  4 

DLIFLC  has  as  cohesive  a  set  of  standards  as  any  U.S.  Government  agency, 
although  they  could  be  more  specific;  as  a  matter  of  fact,  the  Interagency 
Language  Roundtable  (ILR)  which  includes  DLIFLC,  NSA,  CIA  and  FBI,  is  presently 
actively  reviewing  and  will  republish  U.S.  Government  proficiency  standards, 
which,  when  published,  will  be  the  benchmarks  for  DLIFLC  training  objectives. 
DLIFLC  is  playing  an  active  and  leading  role  in  these  ILR  actions,  and  is 
working  in  close  concert  with  all  ILR  participating  agencies  to  improve  U.S. 
Government  standards  as  a  whole.  Some  indicators  of  the  leadership  DLIFLC  is 
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providing  the  language  community  in  the  area  of  language  testing  are  the 
November  1981  National  Language  Proficiency  Testing  Conference,  hosted  by 
DLIFLC,  and  our  active  coordinating  role  in  several  interagency  test  develop¬ 
ment  projects  now  being  pursued. 

Following  publication  of  the  ILR-developed  U.S.  Government  proficiency  standards, 
DLIFLC  will  revise  its  DLIFLC  Memorandum  on  this  subject  to  incorporate  the  spirit, 
thrust  and  intent  of  these  new  standards. 

DLIFLC' s  mission  is  to  provide  general  language  training.  DLIFLC's  mission 
is  not  to  prepare  students  for  a  specific  mission,  but  rather,  general  language 
missions.  DLIFLC's  role  in  the  foreign  language  learning  process  is  to  prepare 
its  students  for  a  wide  variety  of  language-related  occupational  skills;  for 
all  of  these  jobs,  language  is  a  necessary,  but  not  sufficient  qualification 
for  accomplishment  of  the  task.  Language  is  common  to  all  of  these  occupational 
skills.  Even  NSA  has  gone  on  record  stating  that  DLIFLC's  mission  is  to 
prepare  its  students  for  general  mission  tasks,  while  NSA  and  Services  are  to 
provide  specific  job-related  skills  language  training  at  follow-on  MOS  training 
facilities. 
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DLI  STUDENT  ENROLLMENT  AND  INSTRUCTORS 


BY  LANGUAGE  AS  OF  SEPTEMBER  11,  1981 


Language 

Students 

Instructo 

Asian/Middle  East  group: 

Arabic 

186 

35 

Chinese 

196 

37 

Tagalog 

2 

1 

Greek 

31 

8 

Indonesian  (including  Malay) 

7 

3 

Japanese 

10 

2 

Korean 

361 

62 

Persian 

6 

3 

Thai 

3 

3 

Turkish 

36 

11 

Vietnamese 

49 

8 

Total 

887 

173 

Romantic/Germanic  group: 

Albanian 

2 

1 

Dutch 

4 

1 

French 

23 

9 

Hungarian 

7 

3 

German 

304 

55 

Italian 

38 

8 

Norwegian 

4 

1 

Portuguese 

6 

4 

Romanian 

7 

2 

Spanish 

182 

28 

Total 

577 

112 

Slavic  group: 

Russian 

a/1,213 

a/179 

Bulgarian 

5 

3 

Czech 

134 

20 

Polish 

70 

15 

Serbo-Croatian 

8 

6 

Total 

1,430 

223 

Total 

2,894 

b/ 508 

a/Includes  410  students  and  55  faculty  members 

at  Lackland. 

b/In  addition,  there  are  389  secretarial  and  68  management 
staff . 
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PREVIOUS  STUDIES  OF  OPERATIONS 
AND  TRAINING  AT  DLI 

General  Accounting  Office,  Need  To  Improve  Language  Training 
Programs  and  Assignments  for  U.S.  Government  Personnel 
Overseas,  B-176049,  January  22,  1973 

General  Accounting  Office,  Improvement  Needed  in  Language  Train¬ 
ing  and  Assignments  for  U.S.  Personnel  Overseas,  ID-76-19, 

June  16,  1976 

General  Accounting  Office,  Need  to  Improve  Foreign  Language 
Training  Programs  and  Assignments  for  Department  Defense 
Personnel,  ID-76-73,  November  24,  1976 

General  Accounting  Office,  More  Competence  in  Foreign  Languages 
Needed  by  Federal  Personnel  Working  Overseas,  ID-80-031, 

April  15,  1980 

Amy  Training  and  Doctrine  Command,  Defense  Language  Institute 
Assessment,  August  12,  1975 

Defense  Language  Institute,  Accreditation  Self-Study  Report, 

June  1978 

United  States  Amy  Intelligence  School,  Fort  Devens  -  Defense 
Language  Institute,  Foreign  Language  Center,  Joint  Field 
Evaluation,  Far  East  and  Germany,  1979 

Defense  Language  Institute,  Foreign  Language  Center,  Field  Eval¬ 
uation  of  DLIFLC  Graduates,  European  Command,  April  24, 

1981 

President's  Commission  on  Foreign  Language  and  International 
Studies,  Strength  Through  Wisdom:  A  Critique  of  U.S. 
Capability,  November  1979 

Amy  Training  and  Doctrine  Command  Inspector  General,  Report 
of  Findings,  March  2,  1979 

Department  of  the  Amy,  The  Amy  Linguist  Problem,  April  24,  1980 

Defense  Language  Institute,  Special  Program  Review,  Summary 
Report,  December  1980 

Defense  Language  Institute,  Annual  Program  Review,  Summary 
Report,  February  1981 

Defense  Language  Institute,  Foreign  Language  Center,  General 
Officer  Steering  Committee,  Report,  January  30,  1981 
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Army  Training  and  Doctrine  Command,  Defense  Language  Institute, 
Foreign  Language  Center,  Management  Study,  July  1981 

Department  of  the  Army,  MILPERCEN  Linguist  Survey,  1977 
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Language  Skill  Level  Description* 


:B3 

MW 

UMMHQ  HI 

n-s»i  i m 

WrtUa*  iW) 

0 

Noit 

No  practical  speaking  proficiency. 

No  practical  reeding  proficiency. 

No  practical  writing  proficiency. 

1 

Clementary 

Has  suffldMt  control  of  the  writ¬ 
ing  system  to  meet  United  prac¬ 
tical  needs.  Can  produce  all  sym¬ 
bols  in  an  alphabetic  or  syllabic 
writing  system.  Can  writs  sum 
bars  and  dates,  his  own  asms  and 
nationality,  addresses,  etc.  Other 
wise,  ability  to  write  is  limited  to 
simple  lists  sf  common  itsms  or  a 
law  short  sentoneoc.  Spelling  may 
bo  erratic. 

2 

Limited 

Working 

Sufficient  comprehension  to  moot 
routio*  soda)  demands  sad  Uni  tod 
job  requirements.  AM#  to  under¬ 
stand  face-to-face  speech  is  a 
standard  dialect.  daltvarod  at  a 
normal  rate  with  aoma  repetition 
and  rswordisg,  about  eve ryday 
topics  coca  moo  peraoaal  and  family 
a*w»,  well  known  currant  avaota, 
and  routine  office  matter*;  eaa 
follow  the  essentials  of  discussion 
or  spoeeh  at  aa  elementary  level 
on  topics  la  a  special  profaaaioaal 
field. 

Able  to  satisfy  routine  soda!  de¬ 
mands  and  limited  work  require¬ 
ments.  Can  handle  with  confi¬ 
dence,  but  not  with  facility,  most 
social  situations,  including  intro¬ 
ductions  and  casual  conversations 
about  current  events,  se  well  as 
work,  family,  and  autobiographical 
information;  can  handle  United 
work  requirements,  needing  help 
in  handling  say  com  plications  or 
difficulties;  can  get  the  gist  of 
most  conversations  on  nontechni¬ 
cal  subjects  (io.,  t spies  which  re¬ 
quire  no  spsHsHsed  knowledge) 
and  has  s  sp  as  king  vocabulary  suf¬ 
ficient  t«  express  MmasH/herssH 
simply  with  seme  circumlocutions; 
accent,  though  often  quite  faulty, 
is  iotslUgibls;  ess  usually  handle 
elementary  constructions  quite  ac¬ 
curately  but  doss  not  have  thor¬ 
ough  or  coofidest  control  of  the 
grammar. 

Able  to  mad  simple  prose,  la  a 
form  equivalent  to  typescript  or 
printing,  on  subjects  within  a  fa¬ 
miliar  context.  With  extensive  use 
of  a  diadooay  css  get  the  general 
tense  of  routine  business  letters, 
iatsrnstioosl  news  items,  or  arti¬ 
cles  In  technical  fields  within  his/ 
her  competence. 

Can  draft  routine  social  corre¬ 
spondence  sad  meet  limited  pro- 
foarioaaJ  needs,  fa  familiar  with 
the  mechanics  of  the  writing  sys¬ 
tem.  except  in  character  systems 
where  ability  b  United  to  *  small 
stock  of  high-frequency  items. 
Makes  frequent  errors  in  speUiag, 
style,  and  writing  eoevontions. 
AMs  to  write  simple  notes  sad 
draft  routine  social  sad  limited  of 
flee  ■  sonatas.  Material  normally 

dent  writer 
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Language  Skill  Level  Description! —  Continued 


J 

Minimum 

Profes¬ 

sional 

j  Able  to  understand  the  essentials 

J  of  all  speech  m  a  standard  dialect 
including  technical  discussions 
within  a  special  field.  Has  effective 
understanding  of  faee-to-faee 
speech,  delivered  with  normal 
clarity  and  speed  in  a  standard 
dialect,  on  general  topics  and 
areas  of  special  interest;  has  broad 
enough  vocabulary  that  he/ahe 
rarely  has  to  ask  for  paraphrasing 
or  explanation;  can  follow  accu¬ 
rately  the  eceeatisis  of  eon  versa 
tiooe  betweea  educated  native 
speaker*,  reasonably  dear  tele¬ 
phone  calls,  radio  broadcasts,  and 
public  nddrsseea  on  nontechnical 
subjects;  can  understand  without 
difficulty  all  forma  of  standard 
speech  concerning  a  special  prufet 
Moeal  field. 

Able  to  speak  the  language  with 
sufficient  structural  accuracy  and 
vocabulary  to  participate  effec¬ 
tively  in  most  formal  and  informal 
conversations  on  practical,  social, 
and  professional  topics.  Can  dis¬ 
cuss  particular  interest:  and  spe¬ 
cial  fields  of  competence  with  rea¬ 
sonable  ease;  comprehension  is 
quite  complete  for  a  normal  rate 
of  speech;  vocabulary  is  broad 
enough  that  be  rarely  baa  to  grope 
for  a  word;  accent  may  be  obvious¬ 
ly  foreign;  control  of  grammar 
good;  errors  never  interfere  with 
understanding  and  rarely  disturb 
the  native  speaker. 

Able  to  read  standard  newspaper 
items  addressed  to  the  general 
reader,  routine  correspondence, 
reports  and  technical  material  in 
his/her  special  field.  Can  grasp  the 
essentials  of  articles  of  the  stove 
types  without  using  a  dictionary: 
for  accurate  undorstanding.  mod¬ 
erately  frequent  use  of  a  diction¬ 
ary  is  required.  Has  occasional  dif¬ 
ficulty  with  unusually  complex 
structures  and  low  frequency 
idioms. 

Can  draft  official  correspondence 
and  reports  in  a  special  field.  Con¬ 
trol  of  structure,  spelling,  and 
vocabulary  is  adequate  to  convey 
his/her  message  accurately,  but 
style  may  be  quit*  foreign.  All  for¬ 
mal  writing  needs  to  bo  odited  by 
an  educated  native. 

1 

1 

Able  to  understand  all  forma  and 
styles  of  speech  pertinent  to  pro¬ 
fessional  needs.  Able  to  under¬ 
stand  fully  nil  speech  In  all  stand¬ 
ard  dialects  on  any  subjact  rele¬ 
vant  Vo  prof  test  on  al  needs  within 
the  range  of  his/her  experience, 
including  aocal  conversations;  all 
intelligible  broadcasts  and  tele¬ 
phone  calls;  and  all  Usds  of  tech¬ 
nical  rti  sc  notions  and  discourse. 

of  speech  in  some  nonstandard  dia¬ 
lects. 

Able  to  us*  the  language  fluently 
and  accurately  on  all  levels  nor¬ 
mally  pertinent  to  professional 
oeeds.  Can  understand  sad  partic¬ 
ipate  in  any  conversation  within 
the  range  of  his  experience  with  a 
high  degree  of  fluency  and  preci¬ 
sion  of  vocabulary;  would  rarely 
be  taken  for  a  native  speaker;  but 
ean  respond  appropriately  even  in 
unfamiliar  situations;  errors  of 
pronunciation  and  grammar  quite 
rare;  tan  handle  informal  inter¬ 
preting  from  and  Into  the  lan 
gunge. 

Able  to  read  all  styles  and  forma  of 
the  language  pertinent  to  profes¬ 
sional  needs.  With  occasional  use 
of  a  dictionary  can  read  moderate¬ 
ly  difficult  proa*  readily  in  any 
area  directed  to  the  general  read¬ 
er,  and  all  material  in  his/her  spe¬ 
cial  field,  including  official  and  pro¬ 
fessional  documents  and  corre¬ 
spondence;  can  read  reasonably 
legible  handwriting  without  diffi¬ 
culty. 

Can  draft  all  levels  of  prune  perti¬ 
nent  to  professional  needs.  Control 
of  structure,  vocabulary,  nod  spell¬ 
ing  is  broad  and  precise:  sense  of 
style  is  nearly  native.  Error*  are 
rare  and  do  not  interfere  with  on- 
dersUading.  Nevertheless,  drafts 

omenta  need  to  be  edited  b j  no 
educated  votive. 

Native/ 

Bihagua) 

Comprehension  equivalent  to  that 
of  the  educated  native  speaker. 
Able  to  understand  fully  all  forms 
and  \iylea  of  speech  intelligible  to 
the  educated  native  speaker,  in 
duding  a  number  of  regional  and 
illiterate  dialiects.  highly  eolio 
quia!  speech,  and  conversations 
and  discourse  distorted  by  marked 
interference  from  other  noise. 

Speaking  proficiency  equivalent  to 
that  of  an  educatad  native  speak¬ 
er.  Has  complete  fluency  in  the 
language  such  that  his  speech  on 
all  levels  is  fully  accepted  by 
educated  native  speakers  in  all  of 
its  features,  including  breadth  of 
vocabulary  and  idiom,  colloquial 
isms,  and  pertinant  cultural  ref¬ 
erences. 

Rending  proficiency  equivalent  to 
that  of  an  educated  native.  Can 
read  extremely  difficult  and  ab¬ 
stract  prone,  M  well  as  highly  col¬ 
loquial  writings  and  the  classic 
literary  forma  of  the  language. 
With  varying  degrees  of  difficulty 
can  read  all  kinds  of  handwritten 
documents. 

Has  writing  proficiency  equal  to 
that  of  as  educatad  native.  Can 
draft  and  adit  both  formal  and  in¬ 
formal  correspondence,  official 
reports  and  documents,  and  pro¬ 
fessional  art  idea  without  non¬ 
native  error*  of  structure,  apell- 
ing,  style,  or  vocabulary.  Achieve¬ 
ment  of  this  rating  normally  re¬ 
quire*  both  secondary  and  high 
education  in  institutions  where  the 
language  is  the  primary  one  used 
for  Instruction. 
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Language  Proficiency  Code  Key 


Uwl 

C  fCotnprrtrtiwi  LttmU 

StSt«*k%ni 

Jt  'R fading  Ltvtli 

14' t  H>tft»p  L4  \*U 

0 

C-0— No  Practical 
Proficiency 

S-O-No  Practical 
Proficiency 

R-O-No  Practical 
Proficiency 

W-O-No  Practical 
Proficiency 

l 

C-l  Elementary  Prof: 
Understands  most  simple 
questions  and  statements 
on  familiar  topics  when 
spoken  to  very  slowly  and 
distinctly.  These  often 
have  to  be  restated  in 
different  terms  before  he 
understands. 

S-l  Elementary  Prof: 

Asks  and  answers  ques¬ 
tions  on  daily  personal 
needs,  within  a  limited 
vocabulary  and  with  fre¬ 
quent  errors  in  pronuncia¬ 
tion  and  grammar. 

R-l  Elementary  Prof: 
Reads  and  understands 
elementary  lesson  mate¬ 
rial  and  common  public 
signs. 

W  -1  Elementary  Prof: 
Writes  simple  statements 
and  questions  using  a  very 
limited  vocabulary  with 
frequent  errors  in  spelling 
and  structure  that  fre¬ 
quently  obscure  meaning. 

2 

C-2  Limited  Working 

Prof: 

Understands  most  conver- 
.  nation  when  spoken  dis¬ 
tinctly  and  at  a  slower 
than  normal  rate.  Points 
have  to  be  restated  occa¬ 
sionally. 

S-2  Limited  Working 

Prof: 

Converses  intelligibly  but 
without  thorough  control 
of  pronunciation  and 
grammar  within  host 
social  situations,  about 
current  events,  his  work, 
family,  autobiographical 
information  and  non¬ 
technical  subjects. 

R-2  Limited  Working 

Prof: 

Reads  and  understands  in¬ 
termediate  lesson  mate¬ 
rial  or  simple  colloquial 
texts. 

W-2  Limited  Working 

Prof: 

Writes  sentences  on  famil¬ 
iar  topics  using  a  technical 
vocabulary  and  basic 
structure  pattern.  Errors 
in  spelling  and  structure 
occasionally  obscure 
meaning. 

3 

C-3  Minimum  Tech  Prof: 
Understands  general  con¬ 
versation  or  discussioo 
within  his  special  field, 
when  the  rate  of  speech  is 
near  normal. 

S-3  Minimum  Tech  Prof: 
Participates  effectively  in 
all  general  conversation, 
discusses  particular  in¬ 
terests,  and  his  special 

Geld,  without  making 
errors  that  obscure  mcan- 
ing. 

R-3  Minimum  Tech  Prof: 
Reads  and  understands 
material  on  military  and 
international  subjects 
within  bis  field.  Reads  and 
understands  technical 
text  material  at  junior 
high  school  level. 

W-3  Minimum  Tech  Prof: 
Writes  paragraphs  on 
familiar  topics  using  non¬ 
technical  vocabulary  and 
basic  structural  patterns. 
Errors  seldom  obscure 
meaning. 

4 

C~4  Pull  Tech  Prof: 
Understands  any  conver¬ 
sation  within  the  range  of 
his  experience  when  of 
normal  conversational 
spaed. 

S-4  Full  Tech  Prof: 

Speaks  the  language  flu¬ 
ently  and  accurately  on  all 
levels  pertinent  to  mili¬ 
tary  service  needs  with¬ 
out  errors  of  pronuncia¬ 
tion  or  grammar  that 
interfere  with  ease  of 
Understanding. 

R-4  Full  Tech  Prof: 

Reads  high  school  level 
prose  and  material  in  his 
special  field  end  military 
documents  and  corre* 
spondeoee. 

W-4  Full  Tech  Prof: 
Writes  prose  with  suffi¬ 
cient  structural  accuracy 
and  vocabulary  to  satisfy 
pertinent  service  re¬ 
quirements. 

S 

C-S  Native  or  Bilingual 
Proficiency: 

Comprehension  proficien¬ 
cy  equivalent  to  that  of  an 
educated  native  speaker. 

S-S  Native  or  Bilingual 
Proficiency: 

Speaks  with  a  proficiency 
equivalent  to  that  of  an 
educated  native  speaker. 

R-S  Native  or  Bilingual 
Proficiency: 

Reads  at  a  level  of  profi¬ 
ciency  equivalent  to  that 
of  an  educated  native. 

W-3  Native  or  Bilingual 
Proficiency: 

Writes  with  a  Proficiency 
equivalent  to  that  of  an 
educated  native  speaker. 

(961149) 
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